Merge ~rafaeldtinoco/ubuntu/+source/targetcli-fb:focal-merge-upstream-2.1.51 into ubuntu/+source/targetcli-fb:ubuntu/focal-devel

Proposed by Rafael David Tinoco
Status: Merged
Merge reported by: Rafael David Tinoco
Merged at revision: 327f616ba21fff347ec51c57790cd5c28c27256c
Proposed branch: ~rafaeldtinoco/ubuntu/+source/targetcli-fb:focal-merge-upstream-2.1.51
Merge into: ubuntu/+source/targetcli-fb:ubuntu/focal-devel
Diff against target: 1425 lines (+743/-132)
18 files modified
daemon/targetclid (+276/-0)
debian/changelog (+17/-0)
debian/control (+2/-1)
debian/manpages (+1/-0)
debian/patches/0003-Use-etc-rtslib-fb-target-instead-of-etc-target.patch (+5/-5)
debian/rules (+11/-0)
debian/watch (+1/-1)
scripts/targetcli (+162/-12)
setup.py (+4/-1)
systemd/targetclid.service (+13/-0)
systemd/targetclid.socket (+9/-0)
targetcli.8 (+7/-1)
targetcli/ui_backstore.py (+38/-37)
targetcli/ui_node.py (+4/-1)
targetcli/ui_root.py (+55/-18)
targetcli/ui_target.py (+60/-54)
targetcli/version.py (+1/-1)
targetclid.8 (+77/-0)
Reviewer Review Type Date Requested Status
Rafael David Tinoco (community) Approve
Christian Ehrhardt  (community) Approve
Canonical Server Pending
Review via email: mp+379938@code.launchpad.net

Commit message

https://bugs.launchpad.net/ubuntu/+source/python-rtslib-fb/+bug/1854362

comment #24 resolution:

@rafaeldtinoco - please update targetcli-fb
- update targetcli-fb to 2.1.51 - done in debian
- fix d/watch to detect the non *fb* versions - done in debian
- not sure but it might even need an epoch :-/ - changed epoch (1:)
- create DEP8

I'm only missing DEP8, I hope I can do that after freeze date.

* Salsa merge proposals:

  - https://salsa.debian.org/linux-blocks-team/targetcli-fb/-/merge_requests/5
  - https://salsa.debian.org/linux-blocks-team/targetcli-fb/-/merge_requests/4
  - https://salsa.debian.org/linux-blocks-team/targetcli-fb/-/merge_requests/3

To post a comment you must log in.
Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote :

@paelzer,

Added you directly as you were the MIR team requester for this.

This is addressing comment #24 requests.

Comment #26 was preempted by @jamespage already.

Would you mind updating the MIR with a summary after this ?

Thanks a lot!

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Yes, having and extending dep8 tests on this can be done later.
Also I know that you consider doing even more complex tests outside to autopkgtest - as long as we don't skip the testing I'm fine to not do that now.

Taking a look at the MP now ...

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks for submitting to Debian as well, we can get in sync on that later and consider to go ahead for our activities to be in time for 20.04.

The problem of "getting back" is if they don't agree that we need an epoch on this.
IMHO we do as I already outlined on IRC, but you never know ... :-/
At least it is consistent with what we did for the other outdated -fb packages.
So +1 on that aspect.

I'll need to do some builds/tests on my own as no PPA was linked, please bear with me that this takes some time.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

d/watch is ok for the new tarballs

098b93d... by Christian Ehrhardt 

changelog: explain epoch bump

Signed-off-by: Christian Ehrhardt <email address hidden>

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I really think we need to do more for targetclid (https://github.com/open-iscsi/targetcli-fb/commit/7917ef55f6279fa28519d2160c4ec49453c275ec)

- ./usr/bin/targetclid in installed but missing a man page.

- try the new socket activation (daemon is still off by default https://github.com/open-iscsi/targetcli-fb/commit/797778eeb2997d34af7fd3760195f7f8d08470f4)

- changelog needs epoch mentioned (uh I just found there even is a lintian entry for it https://lintian.debian.org/tags/epoch-change-without-comment.html)

- we miss the targetclid.service/socket (https://github.com/open-iscsi/targetcli-fb/commit/ad37f94ae72d0e3d5963ce182e2897c84af9c039)

Summary:
- I'm glad it is still off by default, things will work as before
- We need to package targetclid better
- we need to test and fix targetclid (can be done after FF)

30e7bbd... by Christian Ehrhardt 

d/manpages: install targetclid man page

Signed-off-by: Christian Ehrhardt <email address hidden>

59dfa1f... by Christian Ehrhardt 

d/rules: install and enable targetclid service and socket

Signed-off-by: Christian Ehrhardt <email address hidden>

3d2d5fd... by Christian Ehrhardt 

changelog: targetclid

Signed-off-by: Christian Ehrhardt <email address hidden>

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

+1 to your work so far, but as outlined it missed some bits.

I have added what IMHO was missing, now the review turns around :-)
- Please review that extra additions at [1]
  To do so please pull it there and push it into your branch, then continue the review here.
- I added a review slot for you for that

I'll continue some tests/checks still ... before I give my final +1

[1]: https://code.launchpad.net/~paelzer/ubuntu/+source/targetcli-fb/+git/targetcli-fb/+ref/focal-merge-upstream-2.1.51-extended

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

+1 for the initial part that was proposed.
Please continue on the daemon packaging to be completed.

I updated the bug and listed TODOs [1] and the most recent state summary [2]:

[1]: https://bugs.launchpad.net/ubuntu/+source/ceph-iscsi/+bug/1854362/comments/34
[2]: https://bugs.launchpad.net/ubuntu/+source/ceph-iscsi/+bug/1854362/comments/35

review: Needs Fixing
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I got the service right, this is from an install:

ubuntu@focal-targetcli:~$ sudo apt install targetcli-fb
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
  targetcli-fb
0 upgraded, 1 newly installed, 0 to remove and 9 not upgraded.
Need to get 35.1 kB of archives.
After this operation, 170 kB of additional disk space will be used.
Get:1 http://ppa.launchpad.net/ci-train-ppa-service/3956/ubuntu focal/main amd64 targetcli-fb all 1:2.1.51-0ubuntu1~ppa3 [35.1 kB]
Fetched 35.1 kB in 0s (306 kB/s)
Selecting previously unselected package targetcli-fb.
(Reading database ... 63672 files and directories currently installed.)
Preparing to unpack .../targetcli-fb_1%3a2.1.51-0ubuntu1~ppa3_all.deb ...
Unpacking targetcli-fb (1:2.1.51-0ubuntu1~ppa3) ...
Setting up targetcli-fb (1:2.1.51-0ubuntu1~ppa3) ...
/usr/lib/python3.8/subprocess.py:838: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Created symlink /etc/systemd/system/multi-user.target.wants/targetclid.service → /lib/systemd/system/targetclid.service.
Created symlink /etc/systemd/system/sockets.target.wants/targetclid.socket → /lib/systemd/system/targetclid.socket.
Processing triggers for man-db (2.9.0-2) ...
ubuntu@focal-targetcli:~$ systemctl status targetclid.socket targetclid.service
● targetclid.socket - targetclid socket
     Loaded: loaded (/lib/systemd/system/targetclid.socket; enabled; vendor preset: enabled)
     Active: active (listening) since Thu 2020-02-27 15:21:06 UTC; 5s ago
   Triggers: ● targetclid.service
       Docs: man:targetclid(8)
     Listen: /run/targetclid.sock (Stream)
     CGroup: /system.slice/targetclid.socket

Feb 27 15:21:06 focal-targetcli systemd[1]: Listening on targetclid socket.

● targetclid.service - Targetcli daemon
     Loaded: loaded (/lib/systemd/system/targetclid.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
TriggeredBy: ● targetclid.socket
       Docs: man:targetclid(8)

Feb 27 11:47:08 focal-targetcli systemd[1]: Started Targetcli daemon.
Feb 27 11:47:09 focal-targetcli targetclid[1651]: Warning: Could not load preferences file /root/.targetcli/prefs.bin.
Feb 27 14:08:20 focal-targetcli targetclid[1651]: Signal received, quiting gracefully!
Feb 27 14:08:20 focal-targetcli systemd[1]: Stopping Targetcli daemon...
Feb 27 14:08:20 focal-targetcli systemd[1]: targetclid.service: Succeeded.
Feb 27 14:08:20 focal-targetcli systemd[1]: Stopped Targetcli daemon.

327f616... by Christian Ehrhardt 

changelog: fixup change attribution

Signed-off-by: Christian Ehrhardt <email address hidden>

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

@Rafel
All open TODOs that I found are done now.
+1 from me

Once you have checked the content I added (reverse +1) in [1] please tag and sponsor it.
Let me know then so that we can update the MIR request.

[1]: https://code.launchpad.net/~paelzer/ubuntu/+source/targetcli-fb/+git/targetcli-fb/+ref/focal-merge-upstream-2.1.51-extended

review: Approve
Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote :

> I was going through the upstream changes again one by one.
> at https://github.com/open-iscsi/targetcli-fb/compare/v2.1.fb49...v2.1.51
>
> FYI on reserver/release - new feature https://github.com/open-iscsi/targetcli-
> fb/commit/fa71860b0d819a691683c1fdcb70c255653b5851

That's nice! Important "feature" for this need.

Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote :

I have rebased your branch into mine. I am definitely +1 on all your changes and thankful for you being proactive on them when I couldn't deal with them yet.

- stdout:

dpkg-deb: building package 'targetcli-fb' in '../targetcli-fb_2.1.51-0ubuntu1_all.deb'.
 dpkg-genbuildinfo --build=binary
 dpkg-genchanges -sa --build=binary >../targetcli-fb_2.1.51-0ubuntu1_amd64.changes
dpkg-genchanges: info: binary-only upload (no source code included)
 dpkg-source --after-build .
dpkg-source: info: unapplying 0003-Use-etc-rtslib-fb-target-instead-of-etc-target.patch
dpkg-buildpackage: info: binary-only upload (no source included)
 signfile targetcli-fb_2.1.51-0ubuntu1_amd64.buildinfo
 signfile targetcli-fb_2.1.51-0ubuntu1_amd64.changes

- installation:

(c)rafaeldtinoco@targetclifb:~/.../targetcli-fb$ sudo dpkg -i ../targetcli-fb_2.1.51-0ubuntu1_all.deb
Selecting previously unselected package targetcli-fb.
(Reading database ... 76255 files and directories currently installed.)
Preparing to unpack .../targetcli-fb_2.1.51-0ubuntu1_all.deb ...
Unpacking targetcli-fb (1:2.1.51-0ubuntu1) ...
Setting up targetcli-fb (1:2.1.51-0ubuntu1) ...
/usr/lib/python3.8/subprocess.py:838: RuntimeWarning: line buffering (buffering=1) isn't supported in binary mode, the default buffer size will be used
  self.stdin = io.open(p2cwrite, 'wb', bufsize)
Created symlink /etc/systemd/system/multi-user.target.wants/targetclid.service → /lib/systemd/system/targetclid.service.
Created symlink /etc/systemd/system/sockets.target.wants/targetclid.socket → /lib/systemd/system/targetclid.socket.

- service:

(c)rafaeldtinoco@targetclifb:~/.../targetcli-fb$ systemctl status targetclid.service
● targetclid.service - Targetcli daemon
     Loaded: loaded (/lib/systemd/system/targetclid.service; enabled; vendor preset: enabled)
     Active: inactive (dead)
TriggeredBy: ● targetclid.socket
       Docs: man:targetclid(8)

(c)rafaeldtinoco@targetclifb:~/.../targetcli-fb$ systemctl start targetclid.service

(c)rafaeldtinoco@targetclifb:~/.../targetcli-fb$ systemctl status targetclid.service
● targetclid.service - Targetcli daemon
     Loaded: loaded (/lib/systemd/system/targetclid.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2020-02-27 15:38:36 UTC; 1s ago
TriggeredBy: ● targetclid.socket
       Docs: man:targetclid(8)
   Main PID: 15181 (targetclid)
      Tasks: 3 (limit: 23186)
     Memory: 14.0M
     CGroup: /system.slice/targetclid.service
             └─15181 /usr/bin/python3 /usr/bin/targetclid

I'm replacing my LIO storage machine with this package to make sure its working... and then I'll upload if you are good.

Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote :

Okay so with previous package I had:

https://pastebin.ubuntu.com/p/zR3rQg47kY/

With the new package after restoring config:

https://pastebin.ubuntu.com/p/BW7XXKPqdg/

Client is working:

/> sessions
alias: clufocal01 sid: 1 type: Normal session-state: LOGGED_IN

Will upload this version.

review: Approve
Revision history for this message
Rafael David Tinoco (rafaeldtinoco) wrote :

$ git log -1 --pretty=oneline --tags
327f616 (HEAD -> focal-merge-upstream-2.1.51, tag: upload/1%2.1.51-0ubuntu1, rafaeldtinoco/focal-merge-upstream-2.1.51, pae ....

(c)rafaeldtinoco@targetclifb:~/.../targetcli-fb$ git push pkg upload/1%2.1.51-0ubuntu1
Enumerating objects: 73, done.
Counting objects: 100% (73/73), done.
Delta compression using up to 8 threads
Compressing objects: 100% (51/51), done.
Writing objects: 100% (54/54), 17.01 KiB | 2.83 MiB/s, done.
Total 54 (delta 30), reused 0 (delta 0)
To ssh://git.launchpad.net/~usd-import-team/ubuntu/+source/targetcli-fb
 * [new tag] upload/1%2.1.51-0ubuntu1 -> upload/1%2.1.51-0ubuntu1

(c)rafaeldtinoco@targetclifb:~/.../targetcli-fb$ dput ubuntu ../targetcli-fb_2.1.51-0ubuntu1_source.changes
Checking signature on .changes
gpg: ../targetcli-fb_2.1.51-0ubuntu1_source.changes: Valid signature from A93E0E0AD83C0D0F
Checking signature on .dsc
gpg: ../targetcli-fb_2.1.51-0ubuntu1.dsc: Valid signature from A93E0E0AD83C0D0F
Uploading to ubuntu (via ftp to upload.ubuntu.com):
  Uploading targetcli-fb_2.1.51-0ubuntu1.dsc: done.
  Uploading targetcli-fb_2.1.51.orig.tar.gz: done.
  Uploading targetcli-fb_2.1.51-0ubuntu1.debian.tar.xz: done.
  Uploading targetcli-fb_2.1.51-0ubuntu1_source.buildinfo: done.
  Uploading targetcli-fb_2.1.51-0ubuntu1_source.changes: done.
Successfully uploaded packages.

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1diff --git a/daemon/targetclid b/daemon/targetclid
2new file mode 100644
3index 0000000..fb472dc
4--- /dev/null
5+++ b/daemon/targetclid
6@@ -0,0 +1,276 @@
7+#!/usr/bin/python
8+
9+'''
10+targetclid
11+
12+This file is part of targetcli-fb.
13+Copyright (c) 2019 by Red Hat, Inc.
14+
15+Licensed under the Apache License, Version 2.0 (the "License"); you may
16+not use this file except in compliance with the License. You may obtain
17+a copy of the License at
18+
19+ http://www.apache.org/licenses/LICENSE-2.0
20+
21+Unless required by applicable law or agreed to in writing, software
22+distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
23+WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
24+License for the specific language governing permissions and limitations
25+under the License.
26+'''
27+
28+from __future__ import print_function
29+from targetcli import UIRoot
30+from targetcli import __version__ as targetcli_version
31+from configshell_fb import ConfigShell
32+from os import getuid, getenv, unlink
33+from threading import Thread
34+
35+import sys
36+import socket
37+import struct
38+import fcntl
39+import signal
40+import errno
41+
42+
43+err = sys.stderr
44+
45+class TargetCLI:
46+ def __init__(self):
47+ '''
48+ initializer
49+ '''
50+ # socket for unix communication
51+ self.socket_path = '/var/run/targetclid.sock'
52+ # pid file for defending on multiple daemon runs
53+ self.pid_file = '/var/run/targetclid.pid'
54+
55+ self.NoSignal = True
56+ self.sock = None
57+
58+ # shell console methods
59+ self.shell = ConfigShell(getenv("TARGETCLI_HOME", '~/.targetcli'))
60+ self.con = self.shell.con
61+ self.display = self.shell.con.display
62+ self.render = self.shell.con.render_text
63+
64+ # Handle SIGINT SIGTERM SIGHUP gracefully
65+ signal.signal(signal.SIGINT, self.signal_handler)
66+ signal.signal(signal.SIGTERM, self.signal_handler)
67+ signal.signal(signal.SIGHUP, self.signal_handler)
68+
69+ try:
70+ self.pfd = open(self.pid_file, 'w+')
71+ except IOError as e:
72+ self.display(
73+ self.render(
74+ "opening pidfile failed: %s" %str(e),
75+ 'red'))
76+ sys.exit(1)
77+
78+ self.try_pidfile_lock()
79+
80+ is_root = False
81+ if getuid() == 0:
82+ is_root = True
83+
84+ try:
85+ root_node = UIRoot(self.shell, as_root=is_root)
86+ root_node.refresh()
87+ except Exception as error:
88+ self.display(self.render(str(error), 'red'))
89+ if not is_root:
90+ self.display(self.render("Retry as root.", 'red'))
91+ self.pfd.close()
92+ sys.exit(1)
93+
94+ # Keep track, for later use
95+ self.con_stdout_ = self.con._stdout
96+ self.con_stderr_ = self.con._stderr
97+
98+
99+ def __del__(self):
100+ '''
101+ destructor
102+ '''
103+ if hasattr(self, 'pfd'):
104+ self.pfd.close()
105+
106+
107+ def signal_handler(self, signum, frame):
108+ '''
109+ signal handler
110+ '''
111+ self.NoSignal = False
112+ if self.sock:
113+ self.sock.close()
114+
115+
116+ def try_pidfile_lock(self):
117+ '''
118+ get lock on pidfile, which is to check if targetclid is running
119+ '''
120+ # check if targetclid is already running
121+ lock = struct.pack('hhllhh', fcntl.F_WRLCK, 0, 0, 0, 0, 0)
122+ try:
123+ fcntl.fcntl(self.pfd, fcntl.F_SETLK, lock)
124+ except Exception:
125+ self.display(self.render("targetclid is already running...", 'red'))
126+ self.pfd.close()
127+ sys.exit(1)
128+
129+
130+ def release_pidfile_lock(self):
131+ '''
132+ release lock on pidfile
133+ '''
134+ lock = struct.pack('hhllhh', fcntl.F_UNLCK, 0, 0, 0, 0, 0)
135+ try:
136+ fcntl.fcntl(self.pfd, fcntl.F_SETLK, lock)
137+ except Exception as e:
138+ self.display(
139+ self.render(
140+ "fcntl(UNLCK) on pidfile failed: %s" %str(e),
141+ 'red'))
142+ self.pfd.close()
143+ sys.exit(1)
144+ self.pfd.close()
145+
146+
147+ def client_thread(self, connection):
148+ '''
149+ Handle commands from client
150+ '''
151+ # load the prefs
152+ self.shell.prefs.load()
153+
154+ still_listen = True
155+ # Receive the data in small chunks and retransmit it
156+ while still_listen:
157+ data = connection.recv(65535)
158+ if b'-END@OF@DATA-' in data:
159+ connection.close()
160+ still_listen = False
161+ else:
162+ self.con._stdout = self.con._stderr = f = open("/tmp/data.txt", "w")
163+ try:
164+ # extract multiple commands delimited with '%'
165+ list_data = data.decode().split('%')
166+ for cmd in list_data:
167+ self.shell.run_cmdline(cmd)
168+ except Exception as e:
169+ print(str(e), file=f) # push error to stream
170+
171+ # Restore
172+ self.con._stdout = self.con_stdout_
173+ self.con._stderr = self.con_stderr_
174+ f.close()
175+
176+ with open('/tmp/data.txt', 'r') as f:
177+ output = f.read()
178+ var = struct.pack('i', len(output))
179+ connection.sendall(var) # length of string
180+ if len(output):
181+ connection.sendall(output.encode()) # actual string
182+
183+
184+def usage():
185+ print("Usage: %s [--version|--help]" % sys.argv[0], file=err)
186+ print(" --version\t\tPrint version", file=err)
187+ print(" --help\t\tPrint this information", file=err)
188+ sys.exit(0)
189+
190+
191+def version():
192+ print("%s version %s" % (sys.argv[0], targetcli_version), file=err)
193+ sys.exit(0)
194+
195+
196+def usage_version(cmd):
197+ if cmd in ("help", "--help", "-h"):
198+ usage()
199+
200+ if cmd in ("version", "--version", "-v"):
201+ version()
202+
203+
204+def main():
205+ '''
206+ start targetclid
207+ '''
208+ if len(sys.argv) > 1:
209+ usage_version(sys.argv[1])
210+ print("unrecognized option: %s" % (sys.argv[1]))
211+ sys.exit(-1)
212+
213+ to = TargetCLI()
214+
215+ if getenv('LISTEN_PID'):
216+ # the systemd-activation path, using next available FD
217+ fn = sys.stderr.fileno() + 1
218+ try:
219+ sock = socket.fromfd(fn, socket.AF_UNIX, socket.SOCK_STREAM)
220+ except socket.error as err:
221+ to.display(to.render(err.strerror, 'red'))
222+ sys.exit(1)
223+
224+ # save socket so a signal can clea it up
225+ to.sock = sock
226+ else:
227+ # Make sure file doesn't exist already
228+ try:
229+ unlink(to.socket_path)
230+ except:
231+ pass
232+
233+ # Create a TCP/IP socket
234+ try:
235+ sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
236+ except socket.error as err:
237+ to.display(to.render(err.strerror, 'red'))
238+ sys.exit(1)
239+
240+ # save socket so a signal can clea it up
241+ to.sock = sock
242+
243+ # Bind the socket path
244+ try:
245+ sock.bind(to.socket_path)
246+ except socket.error as err:
247+ to.display(to.render(err.strerror, 'red'))
248+ sys.exit(1)
249+
250+ # Listen for incoming connections
251+ try:
252+ sock.listen(1)
253+ except socket.error as err:
254+ to.display(to.render(err.strerror, 'red'))
255+ sys.exit(1)
256+
257+ while to.NoSignal:
258+ try:
259+ # Wait for a connection
260+ connection, client_address = sock.accept()
261+ except socket.error as err:
262+ if err.errno != errno.EBADF or to.NoSignal:
263+ to.display(to.render(err.strerror, 'red'))
264+ break
265+
266+ thread = Thread(target=to.client_thread, args=(connection,))
267+ thread.start()
268+ try:
269+ thread.join()
270+ except:
271+ to.display(to.render(str(error), 'red'))
272+
273+ to.release_pidfile_lock()
274+
275+ if not to.NoSignal:
276+ to.display(to.render("Signal received, quiting gracefully!", 'green'))
277+ sys.exit(0)
278+ sys.exit(1)
279+
280+
281+if __name__ == "__main__":
282+ main()
283diff --git a/debian/changelog b/debian/changelog
284index 6948d6d..493dc40 100644
285--- a/debian/changelog
286+++ b/debian/changelog
287@@ -1,3 +1,20 @@
288+targetcli-fb (1:2.1.51-0ubuntu1) focal; urgency=medium
289+
290+ [ Rafael David Tinoco ]
291+ * New upstream version 2.1.51
292+ - Upstream changed their versioning scheme in a way that makes the
293+ latest version lower than the previous one therefore we had to add
294+ an epoch bump.
295+ * debian/watch: fix the watch file for upstream versions without fb.
296+
297+ [ Christian Ehrhardt ]
298+ * This now provides an optional daemon targetclid to be used for
299+ operations at scale to retain state between commands in memory.
300+ - d/manpages: install targetclid man page
301+ - d/rules: install and enable targetclid service and socket
302+
303+ -- Rafael David Tinoco <rafaeldtinoco@ubuntu.com> Thu, 27 Feb 2020 02:30:18 +0000
304+
305 targetcli-fb (2.1.fb49-1) unstable; urgency=medium
306
307 * New upstream version 2.1.fb49
308diff --git a/debian/control b/debian/control
309index a1ce30a..9c717c2 100644
310--- a/debian/control
311+++ b/debian/control
312@@ -1,7 +1,8 @@
313 Source: targetcli-fb
314 Section: admin
315 Priority: optional
316-Maintainer: Debian LIO Target Packagers <team+linux-blocks@tracker.debian.org>
317+Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
318+XSBC-Original-Maintainer: Debian LIO Target Packagers <team+linux-blocks@tracker.debian.org>
319 Uploaders: Christophe Vu-Brugier <cvubrugier@fastmail.fm>,
320 Ritesh Raj Sarraf <rrs@debian.org>,
321 Christian Seiler <christian@iwakd.de>
322diff --git a/debian/manpages b/debian/manpages
323index 07dd87b..78f958a 100644
324--- a/debian/manpages
325+++ b/debian/manpages
326@@ -1 +1,2 @@
327 targetcli.8
328+targetclid.8
329diff --git a/debian/patches/0003-Use-etc-rtslib-fb-target-instead-of-etc-target.patch b/debian/patches/0003-Use-etc-rtslib-fb-target-instead-of-etc-target.patch
330index 00f6d3d..e7d0324 100644
331--- a/debian/patches/0003-Use-etc-rtslib-fb-target-instead-of-etc-target.patch
332+++ b/debian/patches/0003-Use-etc-rtslib-fb-target-instead-of-etc-target.patch
333@@ -13,7 +13,7 @@ out of the box.
334
335 --- a/targetcli.8
336 +++ b/targetcli.8
337-@@ -355,7 +355,7 @@
338+@@ -360,7 +360,7 @@
339 Save the current configuration settings to a file, from which settings
340 will be restored if the system is rebooted. By default, this will save
341 the configuration to
342@@ -22,7 +22,7 @@ out of the box.
343 .P
344 This command is executed from the configuration root node.
345 .P
346-@@ -415,7 +415,7 @@
347+@@ -420,7 +420,7 @@
348 to change working path to newly-created nodes. Global settings
349 are user-specific and are saved to ~/.targetcli/ upon exit, unlike
350 other groups, which are system-wide and kept in
351@@ -31,7 +31,7 @@ out of the box.
352 .SS BACKSTORE-SPECIFIC
353 .B attribute
354 .br
355-@@ -453,9 +453,9 @@
356+@@ -458,9 +458,9 @@
357 /iscsi/<target_iqn>/tpgX/acls/<initiator_iqn> configuration node. Set
358 the userid and password for full-feature phase for this ACL.
359 .SH FILES
360@@ -42,10 +42,10 @@ out of the box.
361 +.B /etc/rtslib-fb-target/backup/*
362 .SH ENVIRONMENT
363 .SS TARGETCLI_HOME
364- If set, this variable points to a directory that should be used instead of ~/.targetctl
365+ If set, this variable points to a directory that should be used instead of ~/.targetcli
366 --- a/targetcli/ui_root.py
367 +++ b/targetcli/ui_root.py
368-@@ -33,8 +33,8 @@
369+@@ -34,8 +34,8 @@
370 from .ui_node import UINode
371 from .ui_target import UIFabricModule
372
373diff --git a/debian/rules b/debian/rules
374index 89af47e..dd11732 100755
375--- a/debian/rules
376+++ b/debian/rules
377@@ -2,3 +2,14 @@
378
379 %:
380 dh $@ --with python3 -Spybuild
381+
382+override_dh_installsystemd:
383+ # helper for the cli, not an individual entity, keeping it within main pkg
384+ dh_installsystemd -ptargetcli-fb --name=targetclid --no-restart-on-upgrade --no-start targetclid.service
385+ dh_installsystemd -ptargetcli-fb --name=targetclid --no-restart-on-upgrade targetclid.socket
386+
387+override_dh_install:
388+ # not (yet) installed by upstream but required
389+ cp -v systemd/targetclid.service debian/targetcli-fb.targetclid.service
390+ cp -v systemd/targetclid.socket debian/targetcli-fb.targetclid.socket
391+ dh_install
392diff --git a/debian/watch b/debian/watch
393index c2ed1a0..7a50f79 100644
394--- a/debian/watch
395+++ b/debian/watch
396@@ -1,3 +1,3 @@
397 version=4
398-opts=uversionmangle=s/fb//,filenamemangle=s/.+\/v?(\d\S+)\.tar\.gz/targetcli-fb-$1\.tar\.gz/ \
399+opts=uversionmangle=s/fb//,dversionmangle=s/fb//,filenamemangle=s/.+\/v?(\d\S+)\.tar\.gz/targetcli-fb-$1\.tar\.gz/ \
400 https://github.com/open-iscsi/targetcli-fb/releases .*/v?(\d\S+)\.tar\.gz
401diff --git a/scripts/targetcli b/scripts/targetcli
402index b042ad9..04e5aba 100755
403--- a/scripts/targetcli
404+++ b/scripts/targetcli
405@@ -24,10 +24,22 @@ from os import getuid, getenv
406 from targetcli import UIRoot
407 from rtslib_fb import RTSLibError
408 from configshell_fb import ConfigShell, ExecutionError
409-import sys
410 from targetcli import __version__ as targetcli_version
411
412+import sys
413+import socket
414+import struct
415+import readline
416+import six
417+import fcntl
418+
419 err = sys.stderr
420+# lockfile for serializing multiple targetcli requests
421+lock_file = '/var/run/targetcli.lock'
422+socket_path = '/var/run/targetclid.sock'
423+hints = ['/', 'backstores/', 'iscsi/', 'loopback/', 'vhost/', 'xen-pvscsi/',
424+ 'cd', 'pwd', 'ls', 'set', 'get', 'help', 'refresh', 'status',
425+ 'clearconfig', 'restoreconfig', 'saveconfig', 'exit']
426
427 class TargetCLI(ConfigShell):
428 default_prefs = {'color_path': 'magenta',
429@@ -51,14 +63,16 @@ class TargetCLI(ConfigShell):
430 'auto_save_on_exit': True,
431 'max_backup_files': '10',
432 'auto_add_default_portal': True,
433+ 'auto_use_daemon': False,
434 }
435
436 def usage():
437- print("Usage: %s [--version|--help|CMD]" % sys.argv[0], file=err)
438+ print("Usage: %s [--version|--help|CMD|--disable-daemon]" % sys.argv[0], file=err)
439 print(" --version\t\tPrint version", file=err)
440 print(" --help\t\tPrint this information", file=err)
441 print(" CMD\t\t\tRun targetcli shell command and exit", file=err)
442 print(" <nothing>\t\tEnter configuration shell", file=err)
443+ print(" --disable-daemon\tTurn-off the global auto use daemon flag", file=err)
444 print("See man page for more information.", file=err)
445 sys.exit(-1)
446
447@@ -66,16 +80,153 @@ def version():
448 print("%s version %s" % (sys.argv[0], targetcli_version), file=err)
449 sys.exit(0)
450
451+def usage_version(cmd):
452+ if cmd in ("help", "--help", "-h"):
453+ usage()
454+
455+ if cmd in ("version", "--version", "-v"):
456+ version()
457+
458+def try_op_lock(shell, lkfd):
459+ '''
460+ acquire a blocking lock on lockfile, to serialize multiple requests
461+ '''
462+ try:
463+ fcntl.flock(lkfd, fcntl.LOCK_EX) # wait here until ongoing request is finished
464+ except Exception as e:
465+ shell.con.display(
466+ shell.con.render_text(
467+ "taking lock on lockfile failed: %s" %str(e),
468+ 'red'))
469+ sys.exit(1)
470+
471+def release_op_lock(shell, lkfd):
472+ '''
473+ release blocking lock on lockfile, which can allow other requests process
474+ '''
475+ try:
476+ fcntl.flock(lkfd, fcntl.LOCK_UN) # allow other requests now
477+ except Exception as e:
478+ shell.con.display(
479+ shell.con.render_text(
480+ "unlock on lockfile failed: %s" %str(e),
481+ 'red'))
482+ sys.exit(1)
483+ lkfd.close()
484+
485+def completer(text, state):
486+ options = [x for x in hints if x.startswith(text)]
487+ try:
488+ return options[state]
489+ except IndexError:
490+ return None
491+
492+def call_daemon(shell, req):
493+ try:
494+ sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
495+ except socket.error as err:
496+ shell.con.display(shell.con.render_text(err, 'red'))
497+ sys.exit(1)
498+
499+ try:
500+ sock.connect(socket_path)
501+ except socket.error as err:
502+ shell.con.display(shell.con.render_text(err, 'red'))
503+ shell.con.display(
504+ shell.con.render_text("Currently auto_use_daemon is true, "
505+ "hence please make sure targetclid daemon is running ...\n"
506+ "(or)\nIncase if you wish to turn auto_use_daemon to false "
507+ "then run '#targetcli --disable-daemon'", 'red'))
508+ sys.exit(1)
509+
510+ try:
511+ # send request
512+ sock.sendall(req)
513+ except socket.error as err:
514+ shell.con.display(shell.con.render_text(err, 'red'))
515+ sys.exit(1)
516+
517+ var = sock.recv(4) # get length of data
518+ sending = struct.unpack('i', var)
519+ amount_expected = sending[0]
520+ amount_received = 0
521+
522+ # get the actual data in chunks
523+ while amount_received < amount_expected:
524+ data = sock.recv(1024)
525+ amount_received += len(data)
526+ print(data.decode(), end ="")
527+
528+ sock.send(b'-END@OF@DATA-')
529+ sock.close()
530+ sys.exit(0)
531+
532+def get_arguments(shell):
533+ readline.set_completer(completer)
534+ readline.set_completer_delims('')
535+
536+ if 'libedit' in readline.__doc__:
537+ readline.parse_and_bind("bind ^I rl_complete")
538+ else:
539+ readline.parse_and_bind("tab: complete")
540+
541+ if len(sys.argv) > 1:
542+ command = " ".join(sys.argv[1:])
543+ else:
544+ inputs = []
545+ shell.con.display("targetcli shell version %s\n"
546+ "Entering targetcli batch mode for daemonized approach.\n"
547+ "Enter multiple commands separated by newline and "
548+ "type 'exit' to run them all in one go.\n"
549+ % targetcli_version)
550+ while True:
551+ shell.con.raw_write("/> ")
552+ command = six.moves.input()
553+ if command.lower() == "exit":
554+ break
555+ inputs.append(command)
556+ command = '%'.join(inputs) # delimit multiple commands with '%'
557+
558+ if not command:
559+ sys.exit(1)
560+
561+ usage_version(command);
562+
563+ return command
564+
565 def main():
566 '''
567 Start the targetcli shell.
568 '''
569+ shell = TargetCLI(getenv("TARGETCLI_HOME", '~/.targetcli'))
570+
571+ is_root = False
572 if getuid() == 0:
573 is_root = True
574- else:
575- is_root = False
576
577- shell = TargetCLI(getenv("TARGETCLI_HOME", '~/.targetcli'))
578+ try:
579+ lkfd = open(lock_file, 'w+');
580+ except IOError as e:
581+ shell.con.display(
582+ shell.con.render_text("opening lockfile failed: %s" %str(e),
583+ 'red'))
584+ sys.exit(1)
585+
586+ try_op_lock(shell, lkfd)
587+
588+ use_daemon = False
589+ if shell.prefs['auto_use_daemon']:
590+ use_daemon = True
591+
592+ disable_daemon=False
593+ if len(sys.argv) > 1:
594+ usage_version(sys.argv[1])
595+ if sys.argv[1] in ("disable-daemon", "--disable-daemon"):
596+ disable_daemon=True
597+
598+ if use_daemon and not disable_daemon:
599+ call_daemon(shell, get_arguments(shell).encode())
600+ # does not return
601
602 try:
603 root_node = UIRoot(shell, as_root=is_root)
604@@ -87,14 +238,11 @@ def main():
605 sys.exit(-1)
606
607 if len(sys.argv) > 1:
608- if sys.argv[1] in ("--help", "-h"):
609- usage()
610-
611- if sys.argv[1] in ("--version", "-v"):
612- version()
613-
614 try:
615- shell.run_cmdline(" ".join(sys.argv[1:]))
616+ if disable_daemon:
617+ shell.run_cmdline('set global auto_use_daemon=false')
618+ else:
619+ shell.run_cmdline(" ".join(sys.argv[1:]))
620 except Exception as e:
621 print(str(e), file=sys.stderr)
622 sys.exit(1)
623@@ -117,6 +265,8 @@ def main():
624 shell.log.info("Global pref auto_save_on_exit=true")
625 root_node.ui_command_saveconfig()
626
627+ release_op_lock(shell, lkfd)
628+
629
630 if __name__ == "__main__":
631 main()
632diff --git a/setup.py b/setup.py
633index 7b44304..8dff55e 100755
634--- a/setup.py
635+++ b/setup.py
636@@ -30,7 +30,10 @@ setup(
637 maintainer_email = 'agrover@redhat.com',
638 url = 'http://github.com/open-iscsi/targetcli-fb',
639 packages = ['targetcli'],
640- scripts = ['scripts/targetcli'],
641+ scripts = [
642+ 'scripts/targetcli',
643+ 'daemon/targetclid'
644+ ],
645 classifiers = [
646 "Programming Language :: Python",
647 "Programming Language :: Python :: 3",
648diff --git a/systemd/targetclid.service b/systemd/targetclid.service
649new file mode 100644
650index 0000000..dd1b54c
651--- /dev/null
652+++ b/systemd/targetclid.service
653@@ -0,0 +1,13 @@
654+[Unit]
655+Description=Targetcli daemon
656+Documentation=man:targetclid(8)
657+After=network.target
658+
659+[Service]
660+Type=simple
661+ExecStart=/usr/bin/targetclid
662+Restart=on-failure
663+
664+[Install]
665+WantedBy=multi-user.target
666+Also=targetclid.socket
667diff --git a/systemd/targetclid.socket b/systemd/targetclid.socket
668new file mode 100644
669index 0000000..4730fce
670--- /dev/null
671+++ b/systemd/targetclid.socket
672@@ -0,0 +1,9 @@
673+[Unit]
674+Description=targetclid socket
675+Documentation=man:targetclid(8)
676+
677+[Socket]
678+ListenStream=/var/run/targetclid.sock
679+
680+[Install]
681+WantedBy=sockets.target
682diff --git a/targetcli.8 b/targetcli.8
683index 61f4b64..a73f785 100644
684--- a/targetcli.8
685+++ b/targetcli.8
686@@ -10,6 +10,11 @@ administrator to assign local storage resources backed by either
687 files, volumes, local SCSI devices, or ramdisk, and export them to
688 remote systems via network fabrics, such as iSCSI or FCoE.
689 .P
690+There is a daemon component for targetcli, which will greatly improve
691+the overall execution time taken by targetcli commands at scale. For
692+more details about switching to daemonized mode refer to targetclid(8)
693+man page.
694+.P
695 The configuration layout is tree-based, similar to a filesystem, and
696 is navigated in a similar manner.
697 .SH USAGE
698@@ -458,8 +463,9 @@ the userid and password for full-feature phase for this ACL.
699 .B /etc/target/backup/*
700 .SH ENVIRONMENT
701 .SS TARGETCLI_HOME
702-If set, this variable points to a directory that should be used instead of ~/.targetctl
703+If set, this variable points to a directory that should be used instead of ~/.targetcli
704 .SH SEE ALSO
705+.BR targetclid (8),
706 .BR targetctl (8),
707 .BR tcmu-runner (8)
708 .SH AUTHOR
709diff --git a/targetcli/ui_backstore.py b/targetcli/ui_backstore.py
710index 275d46b..8692f22 100644
711--- a/targetcli/ui_backstore.py
712+++ b/targetcli/ui_backstore.py
713@@ -59,14 +59,14 @@ def human_to_bytes(hsize, kilo=1024):
714 '''
715 This function converts human-readable amounts of bytes to bytes.
716 It understands the following units :
717- - I{B} or no unit present for Bytes
718- - I{k}, I{K}, I{kB}, I{KB} for kB (kilobytes)
719- - I{m}, I{M}, I{mB}, I{MB} for MB (megabytes)
720- - I{g}, I{G}, I{gB}, I{GB} for GB (gigabytes)
721- - I{t}, I{T}, I{tB}, I{TB} for TB (terabytes)
722-
723- Note: The definition of I{kilo} defaults to 1kB = 1024Bytes.
724- Strictly speaking, those should not be called I{kB} but I{kiB}.
725+ - B or no unit present for Bytes
726+ - k, K, kB, KB for kB (kilobytes)
727+ - m, M, mB, MB for MB (megabytes)
728+ - g, G, gB, GB for GB (gigabytes)
729+ - t, T, tB, TB for TB (terabytes)
730+
731+ Note: The definition of kilo defaults to 1kB = 1024Bytes.
732+ Strictly speaking, those should not be called "kB" but "kiB".
733 You can override that with the optional kilo parameter.
734
735 @param hsize: The human-readable version of the Bytes amount to convert
736@@ -286,13 +286,13 @@ class UIBackstore(UINode):
737
738 def ui_command_delete(self, name, save=None):
739 '''
740- Recursively deletes the storage object having the specified I{name}. If
741+ Recursively deletes the storage object having the specified name. If
742 there are LUNs using this storage object, they will be deleted too.
743
744 EXAMPLE
745 =======
746- B{delete mystorage}
747- -------------------
748+ delete mystorage
749+ ----------------
750 Deletes the storage object named mystorage, and all associated LUNs.
751 '''
752 self.assert_root()
753@@ -354,7 +354,7 @@ class UIPSCSIBackstore(UIBackstore):
754 def ui_command_create(self, name, dev):
755 '''
756 Creates a PSCSI storage object, with supplied name and SCSI device. The
757- SCSI device I{dev} can either be a path name to the device, in which
758+ SCSI device "dev" can either be a path name to the device, in which
759 case it is recommended to use the /dev/disk/by-id hierarchy to have
760 consistent naming should your physical SCSI system be modified, or an
761 SCSI device ID in the H:C:T:L format, which is not recommended as SCSI
762@@ -383,17 +383,17 @@ class UIRDMCPBackstore(UIBackstore):
763
764 def ui_command_create(self, name, size, nullio=None, wwn=None):
765 '''
766- Creates an RDMCP storage object. I{size} is the size of the ramdisk.
767+ Creates an RDMCP storage object. "size" is the size of the ramdisk.
768
769 SIZE SYNTAX
770 ===========
771 - If size is an int, it represents a number of bytes.
772 - If size is a string, the following units can be used:
773- - B{B} or no unit present for bytes
774- - B{k}, B{K}, B{kB}, B{KB} for kB (kilobytes)
775- - B{m}, B{M}, B{mB}, B{MB} for MB (megabytes)
776- - B{g}, B{G}, B{gB}, B{GB} for GB (gigabytes)
777- - B{t}, B{T}, B{tB}, B{TB} for TB (terabytes)
778+ - B or no unit present for bytes
779+ - k, K, kB, KB for kB (kilobytes)
780+ - m, M, mB, MB for MB (megabytes)
781+ - g, G, gB, GB for GB (gigabytes)
782+ - t, T, tB, TB for TB (terabytes)
783 '''
784 self.assert_root()
785
786@@ -445,14 +445,14 @@ class UIFileIOBackstore(UIBackstore):
787 def ui_command_create(self, name, file_or_dev, size=None, write_back=None,
788 sparse=None, wwn=None):
789 '''
790- Creates a FileIO storage object. If I{file_or_dev} is a path
791- to a regular file to be used as backend, then the I{size}
792- parameter is mandatory. Else, if I{file_or_dev} is a path to a
793- block device, the size parameter B{must} be ommited. If
794- present, I{size} is the size of the file to be used, I{file}
795- the path to the file or I{dev} the path to a block device. The
796- I{write_back} parameter is a boolean controlling write
797- caching. It is enabled by default. The I{sparse} parameter is
798+ Creates a FileIO storage object. If "file_or_dev" is a path
799+ to a regular file to be used as backend, then the "size"
800+ parameter is mandatory. Else, if "file_or_dev" is a path to a
801+ block device, the size parameter must be omitted. If
802+ present, "size" is the size of the file to be used, "file"
803+ the path to the file or "dev" the path to a block device. The
804+ "write_back" parameter is a boolean controlling write
805+ caching. It is enabled by default. The "sparse" parameter is
806 only applicable when creating a new backing file. It is a
807 boolean stating if the created file should be created as a
808 sparse file (the default), or fully initialized.
809@@ -461,11 +461,11 @@ class UIFileIOBackstore(UIBackstore):
810 ===========
811 - If size is an int, it represents a number of bytes.
812 - If size is a string, the following units can be used:
813- - B{B} or no unit present for bytes
814- - B{k}, B{K}, B{kB}, B{KB} for kB (kilobytes)
815- - B{m}, B{M}, B{mB}, B{MB} for MB (megabytes)
816- - B{g}, B{G}, B{gB}, B{GB} for GB (gigabytes)
817- - B{t}, B{T}, B{tB}, B{TB} for TB (terabytes)
818+ - B or no unit present for bytes
819+ - k, K, kB, KB for kB (kilobytes)
820+ - m, M, mB, MB for MB (megabytes)
821+ - g, G, gB, GB for GB (gigabytes)
822+ - t, T, tB, TB for TB (terabytes)
823 '''
824 self.assert_root()
825
826@@ -557,7 +557,7 @@ class UIBlockBackstore(UIBackstore):
827
828 def ui_command_create(self, name, dev, readonly=None, wwn=None):
829 '''
830- Creates an Block Storage object. I{dev} is the path to the TYPE_DISK
831+ Creates an Block Storage object. "dev" is the path to the TYPE_DISK
832 block device to use.
833 '''
834 self.assert_root()
835@@ -628,11 +628,11 @@ class UIUserBackedBackstore(UIBackstore):
836 ===========
837 - If size is an int, it represents a number of bytes.
838 - If size is a string, the following units can be used:
839- - B{B} or no unit present for bytes
840- - B{k}, B{K}, B{kB}, B{KB} for kB (kilobytes)
841- - B{m}, B{M}, B{mB}, B{MB} for MB (megabytes)
842- - B{g}, B{G}, B{gB}, B{GB} for GB (gigabytes)
843- - B{t}, B{T}, B{tB}, B{TB} for TB (terabytes)
844+ - B or no unit present for bytes
845+ - k, K, kB, KB for kB (kilobytes)
846+ - m, M, mB, MB for MB (megabytes)
847+ - g, G, gB, GB for GB (gigabytes)
848+ - t, T, tB, TB for TB (terabytes)
849 '''
850
851 size = human_to_bytes(size)
852@@ -689,6 +689,7 @@ class UIStorageObject(UIRTSLibNode):
853 'emulate_tpws': ('number', 'If set to 1, enable Thin Provisioning Write Same.'),
854 'emulate_ua_intlck_ctrl': ('number', 'If set to 1, enable Unit Attention Interlock.'),
855 'emulate_write_cache': ('number', 'If set to 1, turn on Write Cache Enable.'),
856+ 'emulate_pr': ('number', 'If set to 1, enable SCSI Reservations.'),
857 'enforce_pr_isids': ('number', 'If set to 1, enforce persistent reservation ISIDs.'),
858 'force_pr_aptpl': ('number', 'If set to 1, force SPC-3 PR Activate Persistence across Target Power Loss operation.'),
859 'fabric_max_sectors': ('number', 'Maximum number of sectors the fabric can transfer at once.'),
860diff --git a/targetcli/ui_node.py b/targetcli/ui_node.py
861index a6982f1..58a70c6 100644
862--- a/targetcli/ui_node.py
863+++ b/targetcli/ui_node.py
864@@ -49,6 +49,9 @@ class UINode(ConfigNode):
865 self.define_config_group_param(
866 'global', 'max_backup_files', 'string',
867 'Max no. of configurations to be backed up in /etc/target/backup/ directory.')
868+ self.define_config_group_param(
869+ 'global', 'auto_use_daemon', 'bool',
870+ 'If true, commands will be sent to targetclid.')
871
872 def assert_root(self):
873 '''
874@@ -95,7 +98,7 @@ class UINode(ConfigNode):
875
876 SEE ALSO
877 ========
878- B{ls}
879+ ls
880 '''
881 description, is_healthy = self.summary()
882 self.shell.log.info("Status for %s: %s" % (self.path, description))
883diff --git a/targetcli/ui_root.py b/targetcli/ui_root.py
884index 38118bd..26815bd 100644
885--- a/targetcli/ui_root.py
886+++ b/targetcli/ui_root.py
887@@ -24,6 +24,7 @@ import re
888 import shutil
889 import stat
890 import filecmp
891+import gzip
892
893 from configshell_fb import ExecutionError
894 from rtslib_fb import RTSRoot
895@@ -62,6 +63,38 @@ class UIRoot(UINode):
896 if fm.wwns == None or any(fm.wwns):
897 UIFabricModule(fm, self)
898
899+ def _compare_files(self, backupfile, savefile):
900+ '''
901+ Compare backfile and saveconfig file
902+ '''
903+ if (os.path.splitext(backupfile)[1] == '.gz'):
904+ try:
905+ with gzip.open(backupfile, 'rb') as fbkp:
906+ fdata_bkp = fbkp.read()
907+ except IOError as e:
908+ self.shell.log.warning("Could not gzip open backupfile %s: %s"
909+ % (backupfile, e.strerror))
910+
911+ else:
912+ try:
913+ with open(backupfile, 'rb') as fbkp:
914+ fdata_bkp = fbkp.read()
915+ except IOError as e:
916+ self.shell.log.warning("Could not open backupfile %s: %s"
917+ % (backupfile, e.strerror))
918+
919+ try:
920+ with open(savefile, 'rb') as f:
921+ fdata = f.read()
922+ except IOError as e:
923+ self.shell.log.warning("Could not open saveconfig file %s: %s"
924+ % (savefile, e.strerror))
925+
926+ if fdata_bkp == fdata:
927+ return True
928+ else:
929+ return False
930+
931 def _save_backups(self, savefile):
932 '''
933 Take backup of config-file if needed.
934@@ -72,29 +105,30 @@ class UIRoot(UINode):
935
936 backup_dir = os.path.dirname(savefile) + "/backup/"
937 backup_name = "saveconfig-" + \
938- datetime.now().strftime("%Y%m%d-%H:%M:%S") + ".json"
939+ datetime.now().strftime("%Y%m%d-%H:%M:%S") + "-json.gz"
940 backupfile = backup_dir + backup_name
941 backup_error = None
942
943 if not os.path.exists(backup_dir):
944 try:
945- os.makedirs(backup_dir);
946+ os.makedirs(backup_dir)
947 except OSError as exe:
948 raise ExecutionError("Cannot create backup directory [%s] %s."
949- % (backup_dir, exc.strerror))
950+ % (backup_dir, exe.strerror))
951
952 # Only save backups if savefile exits
953 if not os.path.exists(savefile):
954 return
955
956 backed_files_list = sorted(glob(os.path.dirname(savefile) + \
957- "/backup/*.json"))
958+ "/backup/saveconfig-*json*"))
959
960 # Save backup if backup dir is empty, or savefile is differnt from recent backup copy
961- if not backed_files_list or not filecmp.cmp(backed_files_list[-1], savefile):
962+ if not backed_files_list or not self._compare_files(backed_files_list[-1], savefile):
963 try:
964- shutil.copy(savefile, backupfile)
965-
966+ with open(savefile, 'rb') as f_in, gzip.open(backupfile, 'wb') as f_out:
967+ shutil.copyfileobj(f_in, f_out)
968+ f_out.flush()
969 except IOError as ioe:
970 backup_error = ioe.strerror or "Unknown error"
971
972@@ -139,7 +173,8 @@ class UIRoot(UINode):
973
974 self.shell.log.info("Configuration saved to %s" % savefile)
975
976- def ui_command_restoreconfig(self, savefile=default_save_file, clear_existing=False):
977+ def ui_command_restoreconfig(self, savefile=default_save_file, clear_existing=False,
978+ target=None, storage_object=None):
979 '''
980 Restores configuration from a file.
981 '''
982@@ -151,7 +186,10 @@ class UIRoot(UINode):
983 self.shell.log.info("Restore file %s not found" % savefile)
984 return
985
986- errors = self.rtsroot.restore_from_file(savefile, clear_existing)
987+ target = self.ui_eval_param(target, 'string', None)
988+ storage_object = self.ui_eval_param(storage_object, 'string', None)
989+ errors = self.rtsroot.restore_from_file(savefile, clear_existing,
990+ target, storage_object)
991
992 self.refresh()
993
994@@ -202,15 +240,15 @@ class UIRoot(UINode):
995 PARAMETERS
996 ==========
997
998- I{action}
999- ---------
1000- The I{action} is one of:
1001- - B{list} gives a short session list
1002- - B{detail} gives a detailed list
1003-
1004- I{sid}
1005+ action
1006 ------
1007- You can specify an I{sid} to only list this one,
1008+ The action is one of:
1009+ - `list`` gives a short session list
1010+ - `detail` gives a detailed list
1011+
1012+ sid
1013+ ---
1014+ You can specify an "sid" to only list this one,
1015 with or without details.
1016
1017 SEE ALSO
1018@@ -283,4 +321,3 @@ class UIRoot(UINode):
1019 indent_print("(no open sessions)", base_steps)
1020 else:
1021 raise ExecutionError("no session found with sid %i" % int(sid))
1022-
1023diff --git a/targetcli/ui_target.py b/targetcli/ui_target.py
1024index 6895b38..e8ba6c6 100644
1025--- a/targetcli/ui_target.py
1026+++ b/targetcli/ui_target.py
1027@@ -34,7 +34,8 @@ from .ui_backstore import complete_path
1028 from .ui_node import UINode, UIRTSLibNode
1029
1030 auth_params = ('userid', 'password', 'mutual_userid', 'mutual_password')
1031-discovery_params = auth_params + ("enable",)
1032+int_params = ('enable',)
1033+discovery_params = auth_params + int_params
1034
1035 class UIFabricModule(UIRTSLibNode):
1036 '''
1037@@ -47,8 +48,12 @@ class UIFabricModule(UIRTSLibNode):
1038 self.refresh()
1039 if self.rtsnode.has_feature('discovery_auth'):
1040 for param in discovery_params:
1041- self.define_config_group_param('discovery_auth',
1042- param, 'string')
1043+ if param in int_params:
1044+ self.define_config_group_param('discovery_auth',
1045+ param, 'number')
1046+ else:
1047+ self.define_config_group_param('discovery_auth',
1048+ param, 'string')
1049 self.refresh()
1050
1051 # Support late params
1052@@ -167,18 +172,18 @@ class UIFabricModule(UIRTSLibNode):
1053
1054 def ui_command_create(self, wwn=None):
1055 '''
1056- Creates a new target. The I{wwn} format depends on the transport(s)
1057- supported by the fabric module. If the I{wwn} is ommited, then a
1058+ Creates a new target. The "wwn" format depends on the transport(s)
1059+ supported by the fabric module. If "wwn" is omitted, then a
1060 target will be created using either a randomly generated WWN of the
1061 proper type, or the first unused WWN in the list of possible WWNs if
1062 one is available. If WWNs are constrained to a list (i.e. for hardware
1063 targets addresses) and all WWNs are in use, the target creation will
1064- fail. Use the B{info} command to get more information abour WWN type
1065+ fail. Use the `info` command to get more information abour WWN type
1066 and possible values.
1067
1068 SEE ALSO
1069 ========
1070- B{info}
1071+ info
1072 '''
1073 self.assert_root()
1074
1075@@ -223,12 +228,12 @@ class UIFabricModule(UIRTSLibNode):
1076
1077 def ui_command_delete(self, wwn):
1078 '''
1079- Recursively deletes the target with the specified I{wwn}, and all
1080+ Recursively deletes the target with the specified wwn, and all
1081 objects hanging under it.
1082
1083 SEE ALSO
1084 ========
1085- B{create}
1086+ create
1087 '''
1088 self.assert_root()
1089 target = Target(self.rtsnode, wwn, mode='lookup')
1090@@ -262,7 +267,7 @@ class UIFabricModule(UIRTSLibNode):
1091 def ui_command_info(self):
1092 '''
1093 Displays information about the fabric module, notably the supported
1094- transports(s) and accepted B{wwn} format(s), as long as supported
1095+ transports(s) and accepted wwn format(s), along with supported
1096 features.
1097 '''
1098 fabric = self.rtsnode
1099@@ -308,13 +313,13 @@ class UIMultiTPGTarget(UIRTSLibNode):
1100 def ui_command_create(self, tag=None):
1101 '''
1102 Creates a new Target Portal Group within the target. The
1103- I{tag} must be a positive integer value, optionally prefaced
1104+ tag must be a positive integer value, optionally prefaced
1105 by 'tpg'. If omitted, the next available Target Portal Group
1106 Tag (TPGT) will be used.
1107
1108 SEE ALSO
1109 ========
1110- B{delete}
1111+ delete
1112 '''
1113 self.assert_root()
1114
1115@@ -351,12 +356,12 @@ class UIMultiTPGTarget(UIRTSLibNode):
1116
1117 def ui_command_delete(self, tag):
1118 '''
1119- Deletes the Target Portal Group with TPGT I{tag} from the target. The
1120- I{tag} must be a positive integer matching an existing TPGT.
1121+ Deletes the Target Portal Group with TPGT "tag" from the target. The
1122+ tag must be a positive integer matching an existing TPGT.
1123
1124 SEE ALSO
1125 ========
1126- B{create}
1127+ create
1128 '''
1129 self.assert_root()
1130 if tag.startswith("tpg"):
1131@@ -384,7 +389,7 @@ class UIMultiTPGTarget(UIRTSLibNode):
1132 @rtype: list of str
1133 '''
1134 if current_param == 'tag':
1135- tags = [child.name[4:] for child in self.children]
1136+ tags = [child.name[3:] for child in self.children]
1137 completions = [tag for tag in tags if tag.startswith(text)]
1138 else:
1139 completions = []
1140@@ -515,7 +520,7 @@ class UITPG(UIRTSLibNode):
1141
1142 SEE ALSO
1143 ========
1144- B{disable status}
1145+ disable status
1146 '''
1147 self.assert_root()
1148 if self.rtsnode.enable:
1149@@ -533,7 +538,7 @@ class UITPG(UIRTSLibNode):
1150
1151 SEE ALSO
1152 ========
1153- B{enable status}
1154+ enable status
1155 '''
1156 self.assert_root()
1157 if self.rtsnode.enable:
1158@@ -582,18 +587,19 @@ class UINodeACLs(UINode):
1159
1160 def ui_command_create(self, wwn, add_mapped_luns=None):
1161 '''
1162- Creates a Node ACL for the initiator node with the specified I{wwn}.
1163- The node's I{wwn} must match the expected WWN Type of the target's
1164+ Creates a Node ACL for the initiator node with the specified wwn.
1165+ The node's wwn must match the expected WWN Type of the target's
1166 fabric module.
1167
1168- If I{add_mapped_luns} is omitted, the global parameter
1169- B{auto_add_mapped_luns} will be used, else B{true} or B{false} are
1170- accepted. If B{true}, then after creating the ACL, mapped LUNs will be
1171- automatically created for all existing LUNs.
1172+ "add_mapped_luns" can be "true" of "false". If true, then
1173+ after creating the ACL, mapped LUNs will be automatically
1174+ created for all existing LUNs. If the parameter is omitted,
1175+ the global parameter "auto_add_mapped_luns" is used.
1176
1177 SEE ALSO
1178 ========
1179- B{delete}
1180+ delete
1181+
1182 '''
1183 self.assert_root()
1184
1185@@ -614,11 +620,11 @@ class UINodeACLs(UINode):
1186
1187 def ui_command_delete(self, wwn):
1188 '''
1189- Deletes the Node ACL with the specified I{wwn}.
1190+ Deletes the Node ACL with the specified wwn.
1191
1192 SEE ALSO
1193 ========
1194- B{create}
1195+ create
1196 '''
1197 self.assert_root()
1198 node_acl = NodeACL(self.tpg, wwn, mode='lookup')
1199@@ -870,14 +876,14 @@ class UINodeACL(UIRTSLibNode):
1200 def ui_command_create(self, mapped_lun, tpg_lun_or_backstore, write_protect=None):
1201 '''
1202 Creates a mapping to one of the TPG LUNs for the initiator referenced
1203- by the ACL. The provided I{tpg_lun_or_backstore} will appear to that
1204- initiator as LUN I{mapped_lun}. If the I{write_protect} flag is set to
1205- B{1}, the initiator will not have write access to the Mapped LUN.
1206+ by the ACL. The provided "tpg_lun_or_backstore" will appear to that
1207+ initiator as LUN "mapped_lun". If the "write_protect" flag is set to
1208+ 1, the initiator will not have write access to the mapped LUN.
1209
1210- A storage object may also be given for the I{tpg_lun_or_backstore} parameter,
1211+ A storage object may also be given for the "tpg_lun_or_backstore" parameter,
1212 in which case the TPG LUN will be created for that backstore before
1213 mapping the LUN to the initiator. If a TPG LUN for the backstore already
1214- exists, the Mapped LUN will map to that TPG LUN.
1215+ exists, the mapped LUN will map to that TPG LUN.
1216
1217 Finally, a path to an existing block device or file can be given. If so,
1218 a storage object of the appropriate type is created with default parameters,
1219@@ -885,7 +891,7 @@ class UINodeACL(UIRTSLibNode):
1220
1221 SEE ALSO
1222 ========
1223- B{delete}
1224+ delete
1225 '''
1226 self.assert_root()
1227 try:
1228@@ -963,11 +969,11 @@ class UINodeACL(UIRTSLibNode):
1229
1230 def ui_command_delete(self, mapped_lun):
1231 '''
1232- Deletes the specified I{mapped_lun}.
1233+ Deletes the specified mapped LUN.
1234
1235 SEE ALSO
1236 ========
1237- B{create}
1238+ create
1239 '''
1240 self.assert_root()
1241 for na in self.rtsnodes:
1242@@ -1086,25 +1092,25 @@ class UILUNs(UINode):
1243 add_mapped_luns=None):
1244 '''
1245 Creates a new LUN in the Target Portal Group, attached to a storage
1246- object. If the I{lun} parameter is omitted, the first available LUN in
1247+ object. If the "lun" parameter is omitted, the first available LUN in
1248 the TPG will be used. If present, it must be a number greater than 0.
1249- Alternatively, the syntax I{lunX} where I{X} is a positive number is
1250+ Alternatively, the syntax "lunX" where "X" is a positive number is
1251 also accepted.
1252
1253- The I{storage_object} may be the path of an existing storage object,
1254- i.e. B{/backstore/pscsi0/mydisk} to reference the B{mydisk} storage
1255- object of the virtual HBA B{pscsi0}. It also may be the path to an
1256+ The "storage_object" may be the path of an existing storage object,
1257+ i.e. "/backstore/pscsi0/mydisk" to reference the "mydisk" storage
1258+ object of the virtual HBA "pscsi0". It also may be the path to an
1259 existing block device or image file, in which case a storage object
1260 will be created for it first, with default parameters.
1261
1262- If I{add_mapped_luns} is omitted, the global parameter
1263- B{auto_add_mapped_luns} will be used, else B{true} or B{false} are
1264- accepted. If B{true}, then after creating the LUN, mapped LUNs will be
1265- automatically created for all existing node ACLs, mapping the new LUN.
1266+ "add_mapped_luns" can be "true" of "false". If true, then
1267+ after creating the ACL, mapped LUNs will be automatically
1268+ created for all existing LUNs. If the parameter is omitted,
1269+ the global parameter "auto_add_mapped_luns" is used.
1270
1271 SEE ALSO
1272 ========
1273- B{delete}
1274+ delete
1275 '''
1276 self.assert_root()
1277
1278@@ -1188,15 +1194,15 @@ class UILUNs(UINode):
1279
1280 def ui_command_delete(self, lun):
1281 '''
1282- Deletes the supplied LUN from the Target Portal Group. The I{lun} must
1283+ Deletes the supplied LUN from the Target Portal Group. "lun" must
1284 be a positive number matching an existing LUN.
1285
1286- Alternatively, the syntax I{lunX} where I{X} is a positive number is
1287+ Alternatively, the syntax "lunX" where "X" is a positive number is
1288 also accepted.
1289
1290 SEE ALSO
1291 ========
1292- B{create}
1293+ create
1294 '''
1295 self.assert_root()
1296 if lun.lower().startswith("lun"):
1297@@ -1303,9 +1309,9 @@ class UIPortals(UINode):
1298
1299 def ui_command_create(self, ip_address=None, ip_port=None):
1300 '''
1301- Creates a Network Portal with specified I{ip_address} and
1302- I{ip_port}. If I{ip_port} is omitted, the default port for
1303- the target fabric will be used. If I{ip_address} is omitted,
1304+ Creates a Network Portal with the specified IP address and
1305+ port. If the port is omitted, the default port for
1306+ the target fabric will be used. If the IP address is omitted,
1307 INADDR_ANY (0.0.0.0) will be used.
1308
1309 Choosing IN6ADDR_ANY (::0) will listen on all IPv6 interfaces
1310@@ -1317,7 +1323,7 @@ class UIPortals(UINode):
1311
1312 SEE ALSO
1313 ========
1314- B{delete}
1315+ delete
1316 '''
1317 self.assert_root()
1318
1319@@ -1379,11 +1385,11 @@ class UIPortals(UINode):
1320
1321 def ui_command_delete(self, ip_address, ip_port):
1322 '''
1323- Deletes the Network Portal with specified I{ip_address} and I{ip_port}.
1324+ Deletes the Network Portal with the specified IP address and port.
1325
1326 SEE ALSO
1327 ========
1328- B{create}
1329+ create
1330 '''
1331 self.assert_root()
1332 portal = NetworkPortal(self.tpg, self._canonicalize_ip(ip_address),
1333diff --git a/targetcli/version.py b/targetcli/version.py
1334index 5e1527f..d29ae93 100644
1335--- a/targetcli/version.py
1336+++ b/targetcli/version.py
1337@@ -15,4 +15,4 @@ License for the specific language governing permissions and limitations
1338 under the License.
1339 '''
1340
1341-__version__ = '2.1.fb49'
1342+__version__ = '2.1.51'
1343diff --git a/targetclid.8 b/targetclid.8
1344new file mode 100644
1345index 0000000..a783091
1346--- /dev/null
1347+++ b/targetclid.8
1348@@ -0,0 +1,77 @@
1349+.TH targetclid 8
1350+.SH NAME
1351+.B targetclid
1352+\- daemon component for targetcli
1353+.SH DESCRIPTION
1354+.B targetclid
1355+is the daemon component of targetcli, which will help retain state of various
1356+configfs object in memory, hence any new request/command can directly use the
1357+in memory objects instead of reconstructing them by parsing through the entire
1358+configfs files again and again for each and every single command. This will
1359+greatly improve the overall execution time taken by targetcli commands at scale.
1360+
1361+.SH USAGE
1362+.B targetclid [cmd]
1363+.br
1364+.B "--help"
1365+for additional usage information.
1366+.br
1367+.B "--version"
1368+for version information.
1369+.SH QUICKSTART & EXAMPLES
1370+.TP
1371+To start using the daemon, one need to enable targetclid socket,
1372+.br
1373+$ systemctl enable targetclid.socket
1374+.TP
1375+If you would like to use the daemonized approach as default method then,
1376+.br
1377+$ targetcli set global auto_use_daemon=true
1378+.br
1379+$ targetcli ls
1380+.TP
1381+You can use batch mode for sending multiple commands in one go,
1382+.br
1383+$ targetcli <hit-enter>
1384+.br
1385+targetcli shell version 2.1.50
1386+.br
1387+Entering targetcli batch mode for daemonized approach.
1388+.br
1389+Enter multiple commands separated by newline and type 'exit' to run them all in one go.
1390+.br
1391+/> ls
1392+.br
1393+/> pwd
1394+.br
1395+/> get global loglevel_file
1396+.br
1397+/> exit
1398+.br
1399+.TP
1400+You can set preference to stop using daemonized mode even when the daemon is not running,
1401+.br
1402+$ targetcli --disable-daemon
1403+.SH FILES
1404+.B /etc/target/saveconfig.json
1405+.br
1406+.B /etc/target/backup/*
1407+.br
1408+.B /var/run/targetclid.sock
1409+.br
1410+.B /var/run/targetclid.pid
1411+.SH ENVIRONMENT
1412+.SS TARGETCLI_HOME
1413+If set, this variable points to a directory that should be used instead of ~/.targetcli
1414+.SH SEE ALSO
1415+.BR targetcli (8),
1416+.BR targetctl (8),
1417+.BR tcmu-runner (8)
1418+.SH AUTHOR
1419+Written by Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
1420+.br
1421+Man page written by Prasanna Kumar Kalever <prasanna.kalever@redhat.com>
1422+.SH REPORTING BUGS
1423+Report bugs via <targetcli-fb-devel@lists.fedorahosted.org>
1424+.br
1425+or <https://github.com/open-iscsi/targetcli-fb/issues>

Subscribers

People subscribed via source and target branches