CORE — GNUnet link layer
The CORE subsystem in GNUnet is responsible for securing link-layer communications between nodes in the GNUnet overlay network. CORE builds on the TRANSPORT subsystem which provides for the actual, insecure, unreliable link-layer communication (for example, via UDP or WLAN), and then adds fundamental security to the connections:
confidentiality with so-called perfect forward secrecy; we use ECDHE (Elliptic-curve Diffie—Hellman) powered by Curve25519 (Curve25519) for the key exchange and then use symmetric encryption, encrypting with both AES-256 (AES-256) and Twofish (Twofish)
authentication is achieved by signing the ephemeral keys using Ed25519 (Ed25519), a deterministic variant of ECDSA (ECDSA)
integrity protection (using SHA-512 (SHA-512) to do encrypt-then-MAC (encrypt-then-MAC))
Replay (replay) protection (using nonces, timestamps, challenge-response, message counters and ephemeral keys)
liveness (keep-alive messages, timeout)
Limitations Limitations ———–
CORE does not perform routing; using CORE it is only possible to communicate with peers that happen to already be "directly" connected with each other. CORE also does not have an API to allow applications to establish such "direct" connections — for this, applications can ask TRANSPORT, but TRANSPORT might not be able to establish a "direct" connection. The TOPOLOGY subsystem is responsible for trying to keep a few "direct" connections open at all times. Applications that need to talk to particular peers should use the CADET subsystem, as it can establish arbitrary "indirect" connections.
Because CORE does not perform routing, CORE must only be used directly by applications that either perform their own routing logic (such as anonymous file-sharing) or that do not require routing, for example because they are based on flooding the network. CORE communication is unreliable and delivery is possibly out-of-order. Applications that require reliable communication should use the CADET service. Each application can only queue one message per target peer with the CORE service at any time; messages cannot be larger than approximately 63 kilobytes. If messages are small, CORE may group multiple messages (possibly from different applications) prior to encryption. If permitted by the application (using the cork option), CORE may delay transmissions to facilitate grouping of multiple small messages. If cork is not enabled, CORE will transmit the message as soon as TRANSPORT allows it (TRANSPORT is responsible for limiting bandwidth and congestion control). CORE does not allow flow control; applications are expected to process messages at line-speed. If flow control is needed, applications should use the CADET service.
When is a peer "connected"?
In addition to the security features mentioned above, CORE also provides
one additional key feature to applications using it, and that is a
limited form of protocol-compatibility checking. CORE distinguishes
between TRANSPORT-level connections (which enable communication with
other peers) and application-level connections. Applications using the
CORE API will (typically) learn about application-level connections from
CORE, and not about TRANSPORT-level connections. When a typical
application uses CORE, it will specify a set of message types (from
gnunet_protocols.h) that it understands. CORE will then notify the
application about connections it has with other peers if and only if
those applications registered an intersecting set of message types with
their CORE service. Thus, it is quite possible that CORE only exposes a
subset of the established direct connections to a particular application
— and different applications running above CORE might see different
sets of connections at the same time.
A special case are applications that do not register a handler for any
message type. CORE assumes that these applications merely want to
monitor connections (or "all" messages via other callbacks) and will
notify those applications about all connections. This is used, for
example, by the
gnunet-core command-line tool to display the active
connections. Note that it is also possible that the TRANSPORT service
has more active connections than the CORE service, as the CORE service
first has to perform a key exchange with connecting peers before
exchanging information about supported message types and notifying
applications about the new connection.
libgnunetcore libgnunetcore ————-
The CORE API (defined in
gnunet_core_service.h) is the basic
messaging API used by P2P applications built using GNUnet. It provides
applications the ability to send and receive encrypted messages to the
peer’s "directly" connected neighbours.
As CORE connections are generally "direct" connections, applications must not assume that they can connect to arbitrary peers this way, as "direct" connections may not always be possible. Applications using CORE are notified about which peers are connected. Creating new "direct" connections must be done using the TRANSPORT API.
The CORE API provides unreliable, out-of-order delivery. While the implementation tries to ensure timely, in-order delivery, both message losses and reordering are not detected and must be tolerated by the application. Most important, the core will NOT perform retransmission if messages could not be delivered.
Note that CORE allows applications to queue one message per connected peer. The rate at which each connection operates is influenced by the preferences expressed by local application as well as restrictions imposed by the other peer. Local applications can express their preferences for particular connections using the "performance" API of the ATS service.
Applications that require more sophisticated transmission capabilities such as TCP-like behavior, or if you intend to send messages to arbitrary remote peers, should use the CADET API.
The typical use of the CORE API is to connect to the CORE service using
GNUNET_CORE_connect, process events from the CORE service (such as
peers connecting, peers disconnecting and incoming messages) and send
messages to connected peers using
Note that applications must cancel pending transmission requests if they
receive a disconnect event for a peer that had a transmission pending;
furthermore, queuing more than one transmission request per peer per
application using the service is not permitted.
The CORE API also allows applications to monitor all communications of the peer prior to encryption (for outgoing messages) or after decryption (for incoming messages). This can be useful for debugging, diagnostics or to establish the presence of cover traffic (for anonymity). As monitoring applications are often not interested in the payload, the monitoring callbacks can be configured to only provide the message headers (including the message type and size) instead of copying the full data stream to the monitoring client.
The init callback of the
GNUNET_CORE_connect function is called with
the hash of the public key of the peer. This public key is used to
identify the peer globally in the GNUnet network. Applications are
encouraged to check that the provided hash matches the hash that they
are using (as theoretically the application may be using a different
configuration file with a different private key, which would result in
hard to find bugs).
As with most service APIs, the CORE API isolates applications from crashes of the CORE service. If the CORE service crashes, the application will see disconnect events for all existing connections. Once the connections are re-established, the applications will be receive matching connect events.
core client-service protocol .. _The-CORE-Client_002dService-Protocol:
The CORE Client-Service Protocol
This section describes the protocol between an application using the CORE service (the client) and the CORE service process itself.
When a client connects to the CORE service, it first sends a
InitMessage which specifies options for the connection and a set of
message type values which are supported by the application. The options
bitmask specifies which events the client would like to be notified
about. The options include:
Peers connecting and disconnecting
All inbound messages (after decryption) with full payload
MessageHeaderof all inbound messages
All outbound messages (prior to encryption) with full payload
MessageHeaderof all outbound messages
Typical applications will only monitor for connection status changes.
The CORE service responds to the
InitMessage with an
InitReplyMessage which contains the peer’s identity. Afterwards,
both CORE and the client can send messages.
The CORE will send
DisconnectNotifyMessages whenever peers connect or disconnect from
the CORE (assuming their type maps overlap with the message types
registered by the client). When the CORE receives a message that matches
the set of message types specified during the
InitMessage (or if
monitoring is enabled in for inbound messages in the options), it sends
NotifyTrafficMessage with the peer identity of the sender and the
decrypted payload. The same message format (except with
GNUNET_MESSAGE_TYPE_CORE_NOTIFY_OUTBOUND for the message type) is
used to notify clients monitoring outbound messages; here, the peer
identity given is that of the receiver.
When a client wants to transmit a message, it first requests a
transmission slot by sending a
SendMessageRequest which specifies
the priority, deadline and size of the message. Note that these values
may be ignored by CORE. When CORE is ready for the message, it answers
SendMessageReady response. The client can then transmit the
payload with a
SendMessage message. Note that the actual message
size in the
SendMessage is allowed to be smaller than the size in
the original request. A client may at any time send a fresh
SendMessageRequest, which then superceeds the previous
SendMessageRequest, which is then no longer valid. The client can
SendMessageRequest the CORE service’s
SendMessageReady message is for as all of these messages contain a
"unique" request ID (based on a counter incremented by the client for
CORE Peer-to-Peer Protocol .. _The-CORE-Peer_002dto_002dPeer-Protocol:
The CORE Peer-to-Peer Protocol
EphemeralKeyMessage creation .. _Creating-the-EphemeralKeyMessage:
Creating the EphemeralKeyMessage
When the CORE service starts, each peer creates a fresh ephemeral (ECC)
public-private key pair and signs the corresponding
EphemeralKeyMessage with its long-term key (which we usually call
the peer’s identity; the hash of the public long term key is what
results in a
struct GNUNET_PeerIdentity in all GNUnet APIs. The
ephemeral key is ONLY used for an ECDHE (Elliptic-curve
exchange by the CORE service to establish symmetric session keys. A peer
will use the same
EphemeralKeyMessage for all peers for
REKEY_FREQUENCY, which is usually 12 hours. After that time, it will
create a fresh ephemeral key (forgetting the old one) and broadcast the
EphemeralKeyMessage to all connected peers, resulting in fresh
symmetric session keys. Note that peers independently decide on when to
discard ephemeral keys; it is not a protocol violation to discard keys
more often. Ephemeral keys are also never stored to disk; restarting a
peer will thus always create a fresh ephemeral key. The use of ephemeral
keys is what provides forward
Just before transmission, the
EphemeralKeyMessage is patched to
reflect the current sender_status, which specifies the current state of
the connection from the point of view of the sender. The possible values
KX_STATE_DOWNInitial value, never used on the network
KX_STATE_KEY_SENTWe sent our ephemeral key, do not know the key of the other peer
KX_STATE_KEY_RECEIVEDThis peer has received a valid ephemeral key of the other peer, but we are waiting for the other peer to confirm it’s authenticity (ability to decode) via challenge-response.
KX_STATE_UPThe connection is fully up from the point of view of the sender (now performing keep-alive)
KX_STATE_REKEY_SENTThe sender has initiated a rekeying operation; the other peer has so far failed to confirm a working connection using the new ephemeral key
Establishing a connection
Peers begin their interaction by sending a
the other peer once the TRANSPORT service notifies the CORE service
about the connection. A peer receiving an
EphemeralKeyMessage with a
status indicating that the sender does not have the receiver’s ephemeral
key, the receiver’s
EphemeralKeyMessage is sent in response.
Additionally, if the receiver has not yet confirmed the authenticity of
the sender, it also sends an (encrypted)
PingMessage with a
challenge (and the identity of the target) to the other peer. Peers
PingMessage respond with an (encrypted)
which includes the challenge. Peers receiving a
the challenge, and if it matches set the connection to
Encryption and Decryption
All functions related to the key exchange and encryption/decryption of
messages can be found in
gnunet-service-core_kx.c (except for the
cryptographic primitives, which are in
util/crypto*.c). Given the
key material from ECDHE, a Key derivation function (Key derivation
used to derive two pairs of encryption and decryption keys for AES-256
and TwoFish, as well as initialization vectors and authentication keys
(for HMAC (HMAC)). The HMAC is
computed over the encrypted payload. Encrypted messages include an
iv_seed and the HMAC in the header.
Each encrypted message in the CORE service includes a sequence number
and a timestamp in the encrypted payload. The CORE service remembers the
largest observed sequence number and a bit-mask which represents which
of the previous 32 sequence numbers were already used. Messages with
sequence numbers lower than the largest observed sequence number minus
32 are discarded. Messages with a timestamp that is less than
REKEY_TOLERANCE off (5 minutes) are also discarded. This of course
means that system clocks need to be reasonably synchronized for peers to
be able to communicate. Additionally, as the ephemeral key changes every
12 hours, a peer would not even be able to decrypt messages older than
Once an encrypted connection has been established, peers begin to exchange type maps. Type maps are used to allow the CORE service to determine which (encrypted) connections should be shown to which applications. A type map is an array of 65536 bits representing the different types of messages understood by applications using the CORE service. Each CORE service maintains this map, simply by setting the respective bit for each message type supported by any of the applications using the CORE service. Note that bits for message types embedded in higher-level protocols (such as MESH) will not be included in these type maps.
Typically, the type map of a peer will be sparse. Thus, the CORE service
attempts to compress its type map using
("deflate") prior to transmission. However, if the compression fails
to compact the map, the map may also be transmitted without compression
GNUNET_MESSAGE_TYPE_CORE_BINARY_TYPE_MAP messages respectively).
Upon receiving a type map, the respective CORE service notifies
applications about the connection to the other peer if they support any
message type indicated in the type map (or no message type at all). If
the CORE service experience a connect or disconnect event from an
application, it updates its type map (setting or unsetting the
respective bits) and notifies its neighbours about the change. The CORE
services of the neighbours then in turn generate connect and disconnect
events for the peer that sent the type map for their respective
applications. As CORE messages may be lost, the CORE service confirms
receiving a type map by sending back a
GNUNET_MESSAGE_TYPE_CORE_CONFIRM_TYPE_MAP. If such a confirmation
(with the correct hash of the type map) is not received, the sender will
retransmit the type map (with exponential back-off).