lsns(8) new command to list Linux namespaces

The namespaces are commonly used way how to isolate global (ipc, mount, net, ...) resource instances. Unfortunately, we have no command line tool to list namespaces. The new command lsns(8) tries to fill this gap.

Examples:

# lsns
        NS TYPE  NPROCS   PID USER   COMMAND
4026531836 pid      276     1 root   /usr/lib/systemd/systemd --system --deserialize 15
4026531837 user     276     1 root   /usr/lib/systemd/systemd --system --deserialize 15
4026531838 uts      276     1 root   /usr/lib/systemd/systemd --system --deserialize 15
4026531839 ipc      276     1 root   /usr/lib/systemd/systemd --system --deserialize 15
4026531840 mnt      269     1 root   /usr/lib/systemd/systemd --system --deserialize 15
4026531857 mnt        1    63 root   kdevtmpfs
4026531963 net      275     1 root   /usr/lib/systemd/systemd --system --deserialize 15
4026532189 mnt        1   545 root   /usr/lib/systemd/systemd-udevd
4026532390 net        1   776 rtkit  /usr/libexec/rtkit-daemon
4026532478 mnt        1   776 rtkit  /usr/libexec/rtkit-daemon
4026532486 mnt        1   847 colord /usr/libexec/colord
4026532518 mnt        3  6500 root   -bash

and list namespace content:

# lsns 4026532518
  PID  PPID USER COMMAND
 6500  6372 root -bash
19572  6500 root └─/usr/bin/mc -P /tmp/mc-root/mc.pwd.6500
19575 19572 root   └─bash -rcfile .bashrc

help output with columns description:

# lsns -h
  
Usage:
 lsns [options] [namespace]
  
List system namespaces.
  
Options:
 -J, --json             use JSON output format
 -l, --list             use list format output
 -n, --noheadings       don't print headings
 -o, --output list      define which output columns to use
 -p, --task pid         print process namespaces
 -r, --raw              use the raw output format
 -u, --notruncate       don't truncate text in columns
 -t, --type name        namespace type (mnt, net, ipc, user, pid, uts)
  
 -h, --help     display this help and exit
 -V, --version  output version information and exit
  
Available columns (for --output):
          NS  namespace identifier (inode number)
        TYPE  kind of namespace
        PATH  path to the namespace
      NPROCS  number of processes in the namespace
         PID  lowers PID in the namespace
        PPID  PPID of the PID
     COMMAND  command line of the PID
         UID  UID of the PID
        USER  username of the PID
  
For more details see lsns(8).

The important detail is that you can see only namespaces accessible from currently mounted /proc filesystem. The lsns(8) is not able to list persistent namespaces without processes where the namespace instance is hold by bind mounts of the /proc/[pid]/ns/[type] files and the output may be affected by unshared PID namespace and unshared /proc (see unshare(8) for more details).

... it will be probably available in util-linux v2.28 (~ January 2016).

logger v2.28

logger is small util to send log messages from command line. It supports (relatively) a new systemd journal as well as classic syslog. The syslog is still de-facto standard for enterprise admins and the latest logger version add support for RFC5424. This RFC introduces "structured-data" and since v2.28 logger is going to support this feature too.

The structured data is parse-able part of the message in format:

 [SD-ID[@digits] SD-PARAM="value" SD-PARAM="value" ...]

for exmaple:

 <13>1 2015-10-01T14:07:59.168662+02:00 ws kzak - - [timeQuality tzKnown="1" isSynced="1" syncAccuracy="218616"] message

is complete message. The message structured data contains one element with ID "timeQuality". This is standardized element, the custom user defined elements have to use "@digits" suffix in the ID name. It's possible to have arbitrary number of the structured data elements.

And now this functionality is completely exported to logger command line to provide control over the elements, v2.28 is going to introduce two new options, --sd-id to specify structured data element ID and --sd-param to specify one SD-PARAM=value pair, for example:

 logger --rfc5424 --sd-id zoo@123                \
                  --sd-param tiger=\"hungry\"    \
                  --sd-param zebra=\"running\"   \
                  --sd-id manager@123            \
                  --sd-param onMeeting=\"yes\"   \
                  "this is message"   

produces:

  <13>1 2015-10-01T14:07:59.168662+02:00 ws kzak - - [timeQuality tzKnown="1" isSynced="1" syncAccuracy="218616"][zoo@123 tiger="hungry" zebra="running"][manager@123 onMeeting="yes"] this is message

message with three SD elements: timeQuality build-in element, zoo@123 and manager@123 user defined elements.

Now all you need is smart server side or log indexing tool that understand RFC5424 (e.g. rsyslog).

what's cooking in sfdisk for v2.28

I have worked on new features for sfdisk during v2.27 stabilisation. The most user visible change is partitions resize/move improvements. Now (in git tree) sfdisk is able to move partition together with data on the partition. Let's imagine you have disk with three partitions:

Device       Start     End Sectors  Size Type
/dev/sdc1     2048  206847  204800  100M Linux filesystem
/dev/sdc2   206848 1230847 1024000  500M Linux filesystem
/dev/sdc3  1230848 2047966  817119  399M Linux filesystem

where the last partition is your ex-girlfriend /home and the first partition is almost full of pictures. Now you want to remove the last unnecessary partition and enlarge the first partition. And you don't want to lost data on the first and second partitions.

(You have backup of all data, because you understand that all operations related to the partition table are risky. Right? :-)

Let's delete the last partition:

  # sfdisk /dev/sdc --delete 3

Now move the second partition to the end of the device. The size of the deleted sdc3 was 817119 sectors. The sfdisk expects input in format

  [+]start[,[+]size[,type]]

where "+num" means offset or size relative to the original partition setting. If any number is unspecified than default is to use the current setting -- it means that "+817119" is enough. (Note that -N 2 means second partition.)

# echo '+817119' | sfdisk /dev/sdc -N 2 --move-data
...
Old situation:
 
Device      Start     End Sectors  Size Type
/dev/sdc1    2048  206847  204800  100M Linux filesystem
/dev/sdc2  206848 1230847 1024000  500M Linux filesystem
 
New situation:                        

Device       Start     End Sectors  Size Type            
/dev/sdc1     2048  206847  204800  100M Linux filesystem
/dev/sdc2  1023967 2047966 1024000  500M Linux filesystem

Data move:                            
 typescript file: /root/sfdisk-sdc2.move
 old start: 206848, new start: 1023967 (move 1024000 sectors)

The important detail is --move-data option, it forces sfdisk after partition table modification copy data from old area to the new offsets. The source and target may overlap (sfdisk copy sectors in backward order if necessary). The file /root/sfdisk-sdc2.move is log with details about the change.

The last step is to enlarge the first partition to use all available free space (space originally used by the second partition). The string ",+" keeps start offset unchanged and the size is specified as "+" (all available space).

# echo ',+' | sfdisk /dev/sdc -N 1
...
Old situation:
 
Device       Start     End Sectors  Size Type
/dev/sdc1     2048  206847  204800  100M Linux filesystem
/dev/sdc2  1023967 2047966 1024000  500M Linux filesystem
 
New situation:
 
Device       Start     End Sectors  Size Type
/dev/sdc1     2048 1023966 1021919  499M Linux filesystem
/dev/sdc2  1023967 2047966 1024000  500M Linux filesystem

And now inform filesystem about a new space:

# resize2fs /dev/sdc1

and test all by mount:

# mount /dev/sdc1 /mnt/A
# mount /dev/sdc2 /mnt/B
 
# lsblk /dev/sdc
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sdc      8:32   0 1000M  0 disk
├─sdc1   8:33   0  499M  0 part /mnt/A
└─sdc2   8:34   0  500M  0 part /mnt/B

lsipc new command to list IPC facilities

Although it seems that System V IPC is obsolete thing, it's still used by many (enterprise) tools like databases, application servers etc.

The classic tool ipcs(1) provides basic overview, unfortunately the output is very fixed and it's impossible to mix the output columns (for example --pid and --creator cannot be used together etc.). It's also very difficult to extend the current implementation without break backward compatibility.

The solution is a new libsmartcols based tool: lsipc(1). It's like many another util-linux ls-like tools, columns are maintains independently, it's easy to extend, user has absolute control on the output and it provides more output formats (including JSON).

The important and unique is the default (--global) output:

# lsipc
RESOURCE DESCRIPTION                            LIMIT   USED  USE%
MSGMNI   Number of message queues               32000      0 0.00%
MSGMAX   Max size of message (bytes)             8192      0 0.00%
MSGMNB   Default max size of queue (bytes)      16384      0 0.00%
SHMMNI   Shared memory segments                  4096     21 0.51%
SHMALL   Shared memory pages                268435456 294049 0.11%
SEMMNS   Total number of semaphores        1024000000      0 0.00%
SEMMNI   Number of Semaphore IDs                32000      0 0.00%

the information about shared memory is provided with COMMAND column, that makes things more human friendly:

# lsipc --shmem
KEY        ID           PERMS OWNER SIZE NATTCH STATUS CTIME  CPID LPID COMMAND
0x6c6c6536 0        rw-------  root   4K      0        Jul16   288  288
0x00000000 327681   rw-------  kzak   4M      2 dest   Jul16  1487  992 gjs /home/kzak/.local/share/gnome-shell/extensions/
0x00000000 393219   rw-------  kzak 512K      2 dest   Jul16  1141 2853 /usr/bin/gnome-shell
0x00000000 2064389  rw-------  kzak 384K      2 dest   Jul17  9443  992 xchat
0x00000000 47611910 rw-------  kzak   8M      2 dest   10:07  2853  804 /usr/lib64/firefox/firefox
0x00000000 2228231  rw-------  kzak 384K      2 dest   Jul17  9443  992 xchat
0x00000000 2195464  rw-------  kzak   2M      2 dest   Jul17  9443  992 xchat
0x00000000 42303503 rw-------  kzak   8M      2 dest   Jul22  1340  992 /usr/bin/gnome-software --gapplication-service
0x00000000 42270736 rw-------  kzak   4M      2 dest   Jul22  1340  992 /usr/bin/gnome-software --gapplication-service
0x00000000 43188243 rw-------  kzak   4M      2 dest   Jul22  8873 1845 /usr/libexec/gnome-terminal-server
0x00000000 33882133 rw-------  kzak 384K      2 dest   Jul20 26049  992 /usr/bin/python3 /usr/bin/hp-systray --force-startup

and you can define your own output:

# lsipc --shmem -o KEY,SIZE,PERMS
KEY        SIZE     PERMS
0x6c6c6536   4K rw-------
0x00000000   4M rw-------
0x00000000 512K rw-------
0x00000000 384K rw-------
0x00000000   8M rw-------
0x00000000 384K rw-------
0x00000000   2M rw-------
0x00000000   8M rw-------
0x00000000   4M rw-------
0x00000000   4M rw-------
0x00000000 384K rw-------

or ask for specific segment:

# lsipc --shmem --id 47611910
Key:                                0x00000000
ID:                                 47611910
Owner:                              kzak
Permissions:                        rw-------
Creator UID:                        1000
Creator user:                       kzak
Creator GID:                        1000
Creator group:                      kzak
UID:                                1000
User name:                          kzak
GID:                                1000
Group name:                         kzak
Last change:                        Thu Jul 23 10:07:46 2015
Segment size:                       8M
Attached processes:                 2
Status:                             dest
Attach time:                        Thu Jul 23 10:07:46 2015
Creator command:                    /usr/lib64/firefox/firefox
Creator PID:                        2853
Last user PID:                      804

and it should be easy to export complex info about IPC to the monitoring tool:

# lsipc --json -o resource,limit,used
{
   "ipclimits": [
      {"resource": "MSGMNI", "limit": "32000", "used": "0"},
      {"resource": "MSGMAX", "limit": "8192", "used": "0"},
      {"resource": "MSGMNB", "limit": "16384", "used": "0"},
      {"resource": "SHMMNI", "limit": "4096", "used": "11"},
      {"resource": "SHMALL", "limit": "268435456", "used": "8097"},
      {"resource": "SEMMNS", "limit": "1024000000", "used": "0"},
      {"resource": "SEMMNI", "limit": "32000", "used": "0"}
   ]
}

lsipc(1) will be available in util-linux v2.27 (probaly August 2015).

sfdisk --json

As promised by the previous blog I have merged support for JSON dump to sfdisk/libfdisk.

# sfdisk --json /dev/sda
{
   "partitiontable": {
      "label": "gpt",
      "id": "E471810B-5954-48E6-B4F0-5D369ADCC514",
      "device": "/dev/sda",
      "unit": "sectors",
      "firstlba": 34,
      "lastlba": 468862094,
      "partitions": [
         {"node": "/dev/sda1", "start": 2048, "size": 409600, "type": "C12A7328-F81F-11D2-BA4B-00A0C93EC93B", "uuid": "2062335E-51B6-49E8-BC3C-4C6D449E6805", "name": "EFI System Partition"},
         {"node": "/dev/sda2", "start": 411648, "size": 409600, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "DF378D5F-B7BA-4887-BAEB-A81E48DE903D"},
         {"node": "/dev/sda3", "start": 821248, "size": 273266688, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "6073277F-87BC-43FF-BCFD-724C4484A63A"},
         {"node": "/dev/sda4", "start": 274087936, "size": 104857600, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "EC5A52E0-FE99-4B38-91E7-B5FF1120E0E5"},
         {"node": "/dev/sda5", "start": 378945536, "size": 73531392, "type": "0FC63DAF-8483-4772-8E79-3D69D8477DE4", "uuid": "11A62EAE-5ECA-4EF5-8BF5-DD18A4115F4A"},
         {"node": "/dev/sda6", "start": 452476928, "size": 16384000, "type": "0657FD6D-A4AB-43C4-84E5-0933C84B4F4F", "uuid": "A13DACFE-6FFB-4B2C-BD89-B91B768A4F77"}
      ]
   }
}

JSON output for basic tools

I have merged into libsmartcols support for JSON format. The feature is currently enabled for lsblk(8), findmnt(8), lslocks(8) and losetup(8). The library already uses structured data, so add a different output format is pretty trivial. For example findmnt(8) and lsblk(8) are able to gather much information and standardized parsable format makes it easy to use it monitoring tools, automatic bug reporting tools, etc.

$ lsblk --json -o name,type,mountpoint                                    
{                                                                         
   "blockdevices": [                                                      
      {"name": "sda", "type": "disk", "mountpoint": null,                 
         "children": [                                                    
            {"name": "sda1", "type": "part", "mountpoint": "/boot/efi"},  
            {"name": "sda2", "type": "part", "mountpoint": "/boot"},      
            {"name": "sda3", "type": "part", "mountpoint": "/home"},      
            {"name": "sda4", "type": "part", "mountpoint": "/"},          
            {"name": "sda5", "type": "part", "mountpoint": "/home/wine"}, 
            {"name": "sda6", "type": "part", "mountpoint": "[SWAP]"}      
         ]                                                                
      },                                                                  
      {"name": "sdb", "type": "disk", "mountpoint": null,                 
         "children": [                                                    
            {"name": "sdb1", "type": "part", "mountpoint": "/home/archive"}
         ]                                                                
      }                                                                   
   ]                                                                      
}                                                                         

                                                                    
$ findmnt --json --submounts /home                                        
{                                                                         
   "filesystems": [                                                       
      {"target": "/home", "source": "/dev/sda3", "fstype": "ext4", "options": "rw,relatime,data=ordered",
         "children": [                                                    
            {"target": "/home/wine", "source": "/dev/sda5", "fstype": "ext4", "options": "rw,relatime,data=ordered"},
            {"target": "/home/archive", "source": "/dev/sdb1", "fstype": "ext4", "options": "rw,relatime,data=ordered"}
         ]                                                                
      }                                                                   
   ]                                                                      
}

For the next Monday my TODO contains "JSON sfdisk/libfdisk dump output"...

resize by sfdisk

The last week released util-linux v2.26.2 contains improved (fixed:-) sfdisk. The most visible change is that the new version supports partition resize and start offset move.

For example 1GiB disk with GPT:

        echo ", +10M" | ./sfdisk -N 1 /dev/sdc
        ...
       
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 206847  204800  100M Linux filesystem
       
        New situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 227327  225280  110M Linux filesystem

the first partition (-N 1) has been enlarged from 100MiB to 110MiB.

You can also move begin of the partition:

        echo "+2M" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 227327  225280  110M Linux filesystem
       
        New situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   6144 231423  225280  110M Linux filesystem

and it's possible to enalarge as much as possible:

       
        echo ", +" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start    End Sectors  Size Type
        /dev/sdc1   6144 231423  225280  110M Linux filesystem
       
        New situation:
 
        Device     Start     End Sectors  Size Type
        /dev/sdc1   6144 2047966 2041823  997M Linux filesystem

the '+' means "all available space". And vice-versa you can reduce the size of the partition; it's possioble to specify the size in absolute numbers (without +/- signs), or in sectors. The next example reduce size to 100MiB and move to old good offset 2048:

       
        echo "2048,100M" | ./sfdisk -N 1 /dev/sdc
        ...
        Old situation:
       
        Device     Start     End Sectors  Size Type
        /dev/sdc1   6144 2047966 2041823  997M Linux filesystem
       
        New situation:
 
        Device     Start    End Sectors  Size Type
        /dev/sdc1   2048 206847  204800  100M Linux filesystem

Note that fdisks are too low-level to care about filesystems when resize, it's all about partition tables only.

persistent namespaces

Today I merged support for persistent namespaces to unshare(1). The persistent namespace does not require any running process with in the namespace and it's possible to enter the namespace by nsenter(1).

For example let's create a new UTS namespace and set a different hostname within the namespace:

  # hostname
  ws
  # touch /root/ns-uts         
  # unshare --uts=/root/ns-uts
  # hostname FooBar
  # exit

Now there is no process in the namespace, try to enter the namespace by --uts=/root/ns-uts reference:

  # nsenter --uts=/root/ns-uts
  # hostname
  FooBar 
  # exit

The reference to the namespace is bind mount to /proc/[pid]/ns/[type], so umount(8) is enough to remove the reference:

  # umount /root/ns-uts

If there is no another reference or any running process within the namespace then the namesapce is destoyed.

It's also possible to create another types of the persistent namespaces (--net, --ipc, ...). Don't forget that if you want to create a persistent mount namespace than the file (--mount=file) has to be on "private" filesystem, for example on Fedora where all is "shared" you have to use:

  # mount --bind /mnt/test /mnt/test
  # mount --make-rprivate /mnt/test
  # touch /mnt/test/my-ns
  # unshare --mount=/mnt/test/my-ns
  ...

Note that PID namespace cannot be without a running process (or more precisely the PID namespace is dead thing after init process (PID 1) termination).

cfdisk extra partition info

I have merged a new cfdisk(8) improvement. It allows to display/hide additional information about the current (selected) partition:

• filesystem type, LABEL, UUID
  
  The important detail is that libblkid gathers the info by seeking on whole-disk device according to partition offsets from partition table -- so /dev/sda[n] devices do not have to exist or you can use cfdisk for disk images (e.g. "cfdisk file.img").
• partition NAME, TYPE (hex/uuid and human readable name)
• mountpoint -- for mounted filesystem as well as for not mounted (then from /etc/fstab)

All you need is press 'x'.  

The change is going to be available in util-linux v2.27 (May/Jun 2015). Thanks to Ondrej Oprala who has worked on this with me.