trAvis - MANAGER

Edit File: PORTING-DBUS1

A few hints on supporting kdbus as backend in your favorite D-Bus library. ~~~ Before you read this, have a look at the DIFFERENCES and GVARIANT_SERIALIZATION texts you find in the same directory where you found this. We invite you to port your favorite D-Bus protocol implementation over to kdbus. However, there are a couple of complexities involved. On kdbus we only speak GVariant marshaling, kdbus clients ignore traffic in dbus1 marshaling. Thus, you need to add a second, GVariant compatible marshaler to your library first. After you have done that: here's the basic principle how kdbus works: You connect to a bus by opening its bus node in /sys/fs/kdbus/. All buses have a device node there, it starts with a numeric UID of the owner of the bus, followed by a dash and a string identifying the bus. The system bus is thus called /sys/fs/kdbus/0-system, and for user buses the device node is /sys/fs/kdbus/1000-user (if 1000 is your user id). (Before we proceed, please always keep a copy of libsystemd next to you, ultimately that's where the details are, this document simply is a rough overview to help you grok things.) CONNECTING To connect to a bus, simply open() its device node and issue the KDBUS_CMD_HELLO call. That's it. Now you are connected. Do not send Hello messages or so (as you would on dbus1), that does not exist for kdbus. The structure you pass to the ioctl will contain a couple of parameters that you need to know, to operate on the bus. There are two flags fields, one indicating features of the kdbus kernel side ("conn_flags"), the other one ("bus_flags") indicating features of the bus owner (i.e. systemd). Both flags fields are 64bit in width. When calling into the ioctl, you need to place your own supported feature bits into these fields. This tells the kernel about the features you support. When the ioctl returns, it will contain the features the kernel supports. If any of the higher 32bit are set on the two flags fields and your client does not know what they mean, it must disconnect. The upper 32bit are used to indicate "incompatible" feature additions on the bus system, the lower 32bit indicate "compatible" feature additions. A client that does not support a "compatible" feature addition can go on communicating with the bus, however a client that does not support an "incompatible" feature must not proceed with the connection. When a client encountes such an "incompatible" feature it should immediately try the next bus address configured in the bus address string. The hello structure also contains another flags field "attach_flags" which indicates metadata that is optionally attached to all incoming messages. You probably want to set KDBUS_ATTACH_NAMES unconditionally in it. This has the effect that all well-known names of a sender are attached to all incoming messages. You need this information to implement matches that match on a message sender name correctly. Of course, you should only request the attachment of as little metadata fields as you need. The kernel will return in the "id" field your unique id. This is a simple numeric value. For compatibility with classic dbus1 simply format this as string and prefix ":1.". The kernel will also return the bloom filter size and bloom filter hash function number used for the signal broadcast bloom filter (see below). The kernel will also return the bus ID of the bus in a 128bit field. The pool size field specifies the size of the memory mapped buffer. After the calling the hello ioctl, you should memory map the kdbus fd. In this memory mapped region, the kernel will place all your incoming messages. SENDING MESSAGES Use the MSG_SEND ioctl to send a message to another peer. The ioctl takes a structure that contains a variety of fields: The flags field corresponds closely to the old dbus1 message header flags field, though the DONT_EXPECT_REPLY field got inverted into EXPECT_REPLY. The dst_id/src_id field contains the unique id of the destination and the sender. The sender field is overridden by the kernel usually, hence you shouldn't fill it in. The destination field can also take the special value KDBUS_DST_ID_BROADCAST for broadcast messages. For messages intended to a well-known name set the field to KDBUS_DST_ID_NAME, and attach the name in a special "items" entry to the message (see below). The payload field indicates the payload. For all dbus traffic it should carry the value 0x4442757344427573ULL. (Which encodes 'DBusDBus'). The cookie field corresponds with the "serial" field of classic dbus1. We simply renamed it here (and extended it to 64bit) since we didn't want to imply the monotonicity of the assignment the way the word "serial" indicates it. When sending a message that expects a reply, you need to set the EXPECT_REPLY flag in the message flag field. In this case you should also fill out the "timeout_ns" value which indicates the timeout in nsec for this call. If the peer does not respond in this time you will get a notification of a timeout. Note that this is also used for security purposes: a single reply messages is only allowed through the bus as long as the timeout has not ended. With this timeout value you hence "open a time window" in which the peer might respond to your request and the policy allows the response to go through. When sending a message that is a reply, you need to fill in the cookie_reply field, which is similar to the reply_serial field of dbus1. Note that a message cannot have EXPECT_REPLY and a reply_serial at the same time! This pretty much explains the ioctl header. The actual payload of the data is now referenced in additional items that are attached to this ioctl header structure at the end. When sending a message, you attach items of the type PAYLOAD_VEC, PAYLOAD_MEMFD, FDS, BLOOM_FILTER, DST_NAME to it: KDBUS_ITEM_PAYLOAD_VEC: contains a pointer + length pair for referencing arbitrary user memory. This is how you reference most of your data. It's a lot like the good old iovec structure of glibc. KDBUS_ITEM_PAYLOAD_MEMFD: for large data blocks it is preferable to send prepared "memfds" (see below) over. This item contains an fd for a memfd plus a size. KDBUS_ITEM_FDS: for sending over fds attach an item of this type with an array of fds. KDBUS_ITEM_BLOOM_FILTER: the calculated bloom filter of this message, only for undirected (broadcast) message. KDBUS_ITEM_DST_NAME: for messages that are directed to a well-known name (instead of a unique name), this item contains the well-known name field. A single message may consists of no, one or more payload items of type PAYLOAD_VEC or PAYLOAD_MEMFD. D-Bus protocol implementations should treat them as a single block that just happens to be split up into multiple items. Some restrictions apply however: The message header in its entirety must be contained in a single PAYLOAD_VEC item. You may only split your message up right in front of each GVariant contained in the payload, as well is immediately before framing of a Gvariant, as well after as any padding bytes if there are any. The padding bytes must be wholly contained in the preceding PAYLOAD_VEC/PAYLOAD_MEMFD item. You may not split up basic types nor arrays of fixed types. The latter is necessary to allow APIs to return direct pointers to linear arrays of numeric values. Examples: The basic types "u", "s", "t" have to be in the same payload item. The array of fixed types "ay", "ai" have to be fully in contained in the same payload item. For an array "as" or "a(si)" the only restriction however is to keep each string individually in an uninterrupted item, to keep the framing of each element and the array in a single uninterrupted item, however the various strings might end up in different items. Note again, that splitting up messages into separate items is up to the implementation. Also note that the kdbus kernel side might merge separate items if it deems this to be useful. However, the order in which items are contained in the message is left untouched. PAYLOAD_MEMFD items allow zero-copy data transfer (see below regarding the memfd concept). Note however that the overhead of mapping these makes them relatively expensive, and only worth the trouble for memory blocks > 512K (this value appears to be quite universal across architectures, as we tested). Thus we recommend sending PAYLOAD_VEC items over for small messages and restore to PAYLOAD_MEMFD items for messages > 512K. Since while building up the message you might not know yet whether it will grow beyond this boundary a good approach is to simply build the message unconditionally in a memfd object. However, when the message is sealed to be sent away check for the size limit. If the size of the message is < 512K, then simply send the data as PAYLOAD_VEC and reuse the memfd. If it is >= 512K, seal the memfd and send it as PAYLOAD_MEMFD, and allocate a new memfd for the next message. RECEIVING MESSAGES Use the MSG_RECV ioctl to read a message from kdbus. This will return an offset into the pool memory map, relative to its beginning. The received message structure more or less follows the structure of the message originally sent. However, certain changes have been made. In the header the src_id field will be filled in. The payload items might have gotten merged and PAYLOAD_VEC items are not used. Instead, you will only find PAYLOAD_OFF and PAYLOAD_MEMFD items. The former contain an offset and size into your memory mapped pool where you find the payload. If during the HELLO ioctl you asked for getting metadata attached to your message, you will find additional KDBUS_ITEM_CREDS, KDBUS_ITEM_PID_COMM, KDBUS_ITEM_TID_COMM, KDBUS_ITEM_TIMESTAMP, KDBUS_ITEM_EXE, KDBUS_ITEM_CMDLINE, KDBUS_ITEM_CGROUP, KDBUS_ITEM_CAPS, KDBUS_ITEM_SECLABEL, KDBUS_ITEM_AUDIT items that contain this metadata. This metadata will be gathered from the sender at the point in time it sends the message. This information is uncached, and since it is appended by the kernel, trustable. The KDBUS_ITEM_SECLABEL item usually contains the SELinux security label, if it is used. After processing the message you need to call the KDBUS_CMD_FREE ioctl, which releases the message from the pool, and allows the kernel to store another message there. Note that the memory used by the pool is ordinary anonymous, swappable memory that is backed by tmpfs. Hence there is no need to copy the message out of it quickly, instead you can just leave it there as long as you need it and release it via the FREE ioctl only after that's done. BLOOM FILTERS The kernel does not understand dbus marshaling, it will not look into the message payload. To allow clients to subscribe to specific subsets of the broadcast matches we employ bloom filters. When broadcasting messages, a bloom filter needs to be attached to the message in a KDBUS_ITEM_BLOOM item (and only for broadcasting messages!). If you don't know what bloom filters are, read up now on Wikipedia. In short: they are a very efficient way how to probabilistically check whether a certain word is contained in a vocabulary. It knows no false negatives, but it does know false positives. The parameters for the bloom filters that need to be included in broadcast message is communicated to userspace as part of the hello response structure (see above). By default it has the parameters m=512 (bits in the filter), k=8 (nr of hash functions). Note however, that this is subject to change in later versions, and userspace implementations must be capable of handling m values between at least m=8 and m=2^32, and k values between at least k=1 and k=32. The underlying hash function is SipHash-2-4. It is used with a number of constant (yet originally randomly generated) 128bit hash keys, more specifically: b9,66,0b,f0,46,70,47,c1,88,75,c4,9c,54,b9,bd,15, aa,a1,54,a2,e0,71,4b,39,bf,e1,dd,2e,9f,c5,4a,3b, 63,fd,ae,be,cd,82,48,12,a1,6e,41,26,cb,fa,a0,c8, 23,be,45,29,32,d2,46,2d,82,03,52,28,fe,37,17,f5, 56,3b,bf,ee,5a,4f,43,39,af,aa,94,08,df,f0,fc,10, 31,80,c8,73,c7,ea,46,d3,aa,25,75,0f,9e,4c,09,29, 7d,f7,18,4b,7b,a4,44,d5,85,3c,06,e0,65,53,96,6d, f2,77,e9,6f,93,b5,4e,71,9a,0c,34,88,39,25,bf,35 When calculating the first bit index into the bloom filter, the SipHash-2-4 hash value is calculated for the input data and the first 16 bytes of the array above as hash key. Of the resulting 8 bytes of output, as many full bytes are taken for the bit index as necessary, starting from the output's first byte. For the second bit index the same hash value is used, continuing with the next unused output byte, and so on. Each time the bytes returned by the hash function are depleted it is recalculated with the next 16 byte hash key from the array above and the same input data. For each message to send across the bus we populate the bloom filter with all possible matchable strings. If a client then wants to subscribe to messages of this type, it simply tells the kernel to test its own calculated bit mask against the bloom filter of each message. More specifically, the following strings are added to the bloom filter of each message that is broadcasted: The string "interface:" suffixed by the interface name The string "member:" suffixed by the member name The string "path:" suffixed by the path name The string "path-slash-prefix:" suffixed with the path name, and also all prefixes of the path name (cut off at "/"), also prefixed with "path-slash-prefix". The string "message-type:" suffixed with the strings "signal", "method_call", "error" or "method_return" for the respective message type of the message. If the first argument of the message is a string, "arg0:" suffixed with the first argument. If the first argument of the message is a string, "arg0-dot-prefix" suffixed with the first argument, and also all prefixes of the argument (cut off at "."), also prefixed with "arg0-dot-prefix". If the first argument of the message is a string, "arg0-slash-prefix" suffixed with the first argument, and also all prefixes of the argument (cut off at "/"), also prefixed with "arg0-slash-prefix". Similar for all further arguments that are strings up to 63, for the arguments and their "dot" and "slash" prefixes. On the first argument that is not a string, addition to the bloom filter should be stopped however. (Note that the bloom filter does not contain sender nor receiver names!) When a client wants to subscribe to messages matching a certain expression, it should calculate the bloom mask following the same algorithm. The kernel will then simply test the mask against the attached bloom filters. Note that bloom filters are probabilistic, which means that clients might get messages they did not expect. Your bus protocol implementation must be capable of dealing with these unexpected messages (which it needs to anyway, given that transfers are relatively unrestricted on kdbus and people can send you all kinds of non-sense). If a client connects to a bus whose bloom filter metrics (i.e. filter size and number of hash functions) are outside of the range the client supports it must immediately disconnect and continue connection with the next bus address of the bus connection string. INSTALLING MATCHES To install matches for broadcast messages, use the KDBUS_CMD_ADD_MATCH ioctl. It takes a structure that contains an encoded match expression, and that is followed by one or more items, which are combined in an AND way. (Meaning: a message is matched exactly when all items attached to the original ioctl struct match). To match against other user messages add a KDBUS_ITEM_BLOOM item in the match (see above). Note that the bloom filter does not include matches to the sender names. To additionally check against sender names, use the KDBUS_ITEM_ID (for unique id matches) and KDBUS_ITEM_NAME (for well-known name matches) item types. To match against kernel generated messages (see below) you should add items of the same type as the kernel messages include, i.e. KDBUS_ITEM_NAME_ADD, KDBUS_ITEM_NAME_REMOVE, KDBUS_ITEM_NAME_CHANGE, KDBUS_ITEM_ID_ADD, KDBUS_ITEM_ID_REMOVE and fill them out. Note however, that you have some wildcards in this case, for example the .id field of KDBUS_ITEM_ID_ADD/KDBUS_ITEM_ID_REMOVE structures may be set to 0 to match against any id addition/removal. Note that dbus match strings do no map 1:1 to these ioctl() calls. In many cases (where the match string is "underspecified") you might need to issue up to six different ioctl() calls for the same match. For example, the empty match (which matches against all messages), would translate into one KDBUS_ITEM_BLOOM ioctl, one KDBUS_ITEM_NAME_ADD, one KDBUS_ITEM_NAME_CHANGE, one KDBUS_ITEM_NAME_REMOVE, one KDBUS_ITEM_ID_ADD and one KDBUS_ITEM_ID_REMOVE. When creating a match, you may attach a "cookie" value to them, which is used for deleting this match again. The cookie can be selected freely by the client. When issuing KDBUS_CMD_REMOVE_MATCH, simply pass the same cookie as before and all matches matching the same "cookie" value will be removed. This is particularly handy for the case where multiple ioctl()s are added for a single match strings. MEMFDS memfds may be sent across kdbus via KDBUS_ITEM_PAYLOAD_MEMFD items attached to messages. If this is done, the data included in the memfd is considered part of the payload stream of a message, and are treated the same way as KDBUS_ITEM_PAYLOAD_VEC by the receiving side. It is possible to interleave KDBUS_ITEM_PAYLOAD_MEMFD and KDBUS_ITEM_PAYLOAD_VEC items freely, by the reader they will be considered a single stream of bytes in the order these items appear in the message, that just happens to be split up at various places (regarding rules how they may be split up, see above). The kernel will refuse taking KDBUS_ITEM_PAYLOAD_MEMFD items that refer to memfds that are not sealed. Note that sealed memfds may be unsealed again if they are not mapped you have the only fd reference to them. Alternatively to sending memfds as KDBUS_ITEM_PAYLOAD_MEMFD items (where they are just a part of the payload stream of a message) you can also simply attach any memfd to a message using KDBUS_ITEM_PAYLOAD_FDS. In this case, the memfd contents is not considered part of the payload stream of the message, but simply fds like any other, that happen to be attached to the message. MESSAGES FROM THE KERNEL A couple of messages previously generated by the dbus1 bus driver are now generated by the kernel. Since the kernel does not understand the payload marshaling, they are generated by the kernel in a different format. This is indicated with the "payload type" field of the messages set to 0. Library implementations should take these messages and synthesize traditional driver messages for them on reception. More specifically: Instead of the NameOwnerChanged, NameLost, NameAcquired signals there are kernel messages containing KDBUS_ITEM_NAME_ADD, KDBUS_ITEM_NAME_REMOVE, KDBUS_ITEM_NAME_CHANGE, KDBUS_ITEM_ID_ADD, KDBUS_ITEM_ID_REMOVE items are generated (each message will contain exactly one of these items). Note that in libsystemd we have obsoleted NameLost/NameAcquired messages, since they are entirely redundant to NameOwnerChanged. This library will hence only synthesize NameOwnerChanged messages from these kernel messages, and never generate NameLost/NameAcquired. If your library needs to stay compatible to the old dbus1 userspace, you possibly might need to synthesize both a NameOwnerChanged and NameLost/NameAcquired message from the same kernel message. When a method call times out, a KDBUS_ITEM_REPLY_TIMEOUT message is generated. This should be synthesized into a method error reply message to the original call. When a method call fails because the peer terminated the connection before responding, a KDBUS_ITEM_REPLY_DEAD message is generated. Similarly, it should be synthesized into a method error reply message. For synthesized messages we recommend setting the cookie field to (uint32_t) -1 (and not (uint64_t) -1!), so that the cookie is not 0 (which the dbus1 spec does not allow), but clearly recognizable as synthetic. Note that the KDBUS_ITEM_NAME_XYZ messages will actually inform you about all kinds of names, including activatable ones. Classic dbus1 NameOwnerChanged messages OTOH are only generated when a name is really acquired on the bus and not just simply activatable. This means you must explicitly check for the case where an activatable name becomes acquired or an acquired name is lost and returns to be activatable. NAME REGISTRY To acquire names on the bus, use the KDBUS_CMD_NAME_ACQUIRE ioctl(). It takes a flags field similar to dbus1's RequestName() bus driver call, however the NO_QUEUE flag got inverted into a QUEUE flag instead. To release a previously acquired name use the KDBUS_CMD_NAME_RELEASE ioctl(). To list acquired names use the KDBUS_CMD_CONN_INFO ioctl. It may be used to list unique names, well known names as well as activatable names and clients currently queuing for ownership of a well-known name. The ioctl will return an offset into the memory pool. After reading all the data you need, you need to release this via the KDBUS_CMD_FREE ioctl(), similar how you release a received message. CREDENTIALS kdbus can optionally attach various kinds of metadata about the sender at the point of time of sending ("credentials") to messages, on request of the receiver. This is both supported on directed and undirected (broadcast) messages. The metadata to attach is selected at time of the HELLO ioctl of the receiver via a flags field (see above). Note that clients must be able to handle that messages contain more metadata than they asked for themselves, to simplify implementation of broadcasting in the kernel. The receiver should not rely on this data to be around though, even though it will be correct if it happens to be attached. In order to avoid programming errors in applications, we recommend though not passing this data on to clients that did not explicitly ask for it. Credentials may also be queried for a well-known or unique name. Use the KDBUS_CMD_CONN_INFO for this. It will return an offset to the pool area again, which will contain the same credential items as messages have attached. Note that when issuing the ioctl, you can select a different set of credentials to gather, than what was originally requested for being attached to incoming messages. Credentials are always specific to the sender's domain that was current at the time of sending, and of the process that opened the bus connection at the time of opening it. Note that this latter data is cached! POLICY The kernel enforces only very limited policy on names. It will not do access filtering by userspace payload, and thus not by interface or method name. This ultimately means that most fine-grained policy enforcement needs to be done by the receiving process. We recommend using PolicyKit for any more complex checks. However, libraries should make simple static policy decisions regarding privileged/unprivileged method calls easy. We recommend doing this by enabling KDBUS_ATTACH_CAPS and KDBUS_ATTACH_CREDS for incoming messages, and then discerning client access by some capability, or if sender and receiver UIDs match. BUS ADDRESSES When connecting to kdbus use the "kernel:" protocol prefix in DBus address strings. The device node path is encoded in its "path=" parameter. Client libraries should use the following connection string when connecting to the system bus: kernel:path=/sys/fs/kdbus/0-system/bus;unix:path=/var/run/dbus/system_bus_socket This will ensure that kdbus is preferred over the legacy AF_UNIX socket, but compatibility is kept. For the user bus use: kernel:path=/sys/fs/kdbus/$UID-user/bus;unix:path=$XDG_RUNTIME_DIR/bus With $UID replaced by the callers numer user ID, and $XDG_RUNTIME_DIR following the XDG basedir spec. Of course the $DBUS_SYSTEM_BUS_ADDRESS and $DBUS_SESSION_BUS_ADDRESS variables should still take precedence. DBUS SERVICE FILES Activatable services for kdbus may not use classic dbus1 service activation files. Instead, programs should drop in native systemd .service and .busname unit files, so that they are treated uniformly with other types of units and activation of the system. Note that this results in a major difference to classic dbus1: activatable bus names can be established at any time in the boot process. This is unlike dbus1 where activatable names are unconditionally available as long as dbus-daemon is running. Being able to control when activatable names are established is essential to allow usage of kdbus during early boot and in initrds, without the risk of triggering services too early. DISCLAIMER This all is so far just the status quo. We are putting this together, because we are quite confident that further API changes will be smaller, but to make this very clear: this is all subject to change, still! We invite you to port over your favorite dbus library to this new scheme, but please be prepared to make minor changes when we still change these interfaces!