.. _CORE-Subsystem-Dev: .. index:: double: CORE; subsystem CORE ==== The CORE API (defined in ``gnunet_core_service.h``) is the basic messaging API used by P2P applications built using GNUnet. It provides applications the ability to send and receive encrypted messages to the peer's \"directly\" connected neighbours. As CORE connections are generally \"direct\" connections, applications must not assume that they can connect to arbitrary peers this way, as \"direct\" connections may not always be possible. Applications using CORE are notified about which peers are connected. Creating new \"direct\" connections must be done using the TRANSPORT API. The CORE API provides unreliable, out-of-order delivery. While the implementation tries to ensure timely, in-order delivery, both message losses and reordering are not detected and must be tolerated by the application. Most important, the core will NOT perform retransmission if messages could not be delivered. Note that CORE allows applications to queue one message per connected peer. The rate at which each connection operates is influenced by the preferences expressed by local application as well as restrictions imposed by the other peer. Local applications can express their preferences for particular connections using the \"performance\" API of the ATS service. Applications that require more sophisticated transmission capabilities such as TCP-like behavior, or if you intend to send messages to arbitrary remote peers, should use the CADET API. The typical use of the CORE API is to connect to the CORE service using ``GNUNET_CORE_connect``, process events from the CORE service (such as peers connecting, peers disconnecting and incoming messages) and send messages to connected peers using ``GNUNET_CORE_notify_transmit_ready``. Note that applications must cancel pending transmission requests if they receive a disconnect event for a peer that had a transmission pending; furthermore, queuing more than one transmission request per peer per application using the service is not permitted. The CORE API also allows applications to monitor all communications of the peer prior to encryption (for outgoing messages) or after decryption (for incoming messages). This can be useful for debugging, diagnostics or to establish the presence of cover traffic (for anonymity). As monitoring applications are often not interested in the payload, the monitoring callbacks can be configured to only provide the message headers (including the message type and size) instead of copying the full data stream to the monitoring client. The init callback of the ``GNUNET_CORE_connect`` function is called with the hash of the public key of the peer. This public key is used to identify the peer globally in the GNUnet network. Applications are encouraged to check that the provided hash matches the hash that they are using (as theoretically the application may be using a different configuration file with a different private key, which would result in hard to find bugs). As with most service APIs, the CORE API isolates applications from crashes of the CORE service. If the CORE service crashes, the application will see disconnect events for all existing connections. Once the connections are re-established, the applications will be receive matching connect events. core client-service protocol .. _The-CORE-Client_002dService-Protocol: The CORE Client-Service Protocol -------------------------------- This section describes the protocol between an application using the CORE service (the client) and the CORE service process itself. .. _Setup2: Setup2 ^^^^^^ When a client connects to the CORE service, it first sends a ``InitMessage`` which specifies options for the connection and a set of message type values which are supported by the application. The options bitmask specifies which events the client would like to be notified about. The options include: **GNUNET_CORE_OPTION_NOTHING** No notifications **GNUNET_CORE_OPTION_STATUS_CHANGE** Peers connecting and disconnecting **GNUNET_CORE_OPTION_FULL_INBOUND** All inbound messages (after decryption) with full payload **GNUNET_CORE_OPTION_HDR_INBOUND** Just the ``MessageHeader`` of all inbound messages **GNUNET_CORE_OPTION_FULL_OUTBOUND** All outbound messages (prior to encryption) with full payload **GNUNET_CORE_OPTION_HDR_OUTBOUND** Just the ``MessageHeader`` of all outbound messages Typical applications will only monitor for connection status changes. The CORE service responds to the ``InitMessage`` with an ``InitReplyMessage`` which contains the peer's identity. Afterwards, both CORE and the client can send messages. .. _Notifications: Notifications ^^^^^^^^^^^^^ The CORE will send ``ConnectNotifyMessage``\ s and ``DisconnectNotifyMessage``\ s whenever peers connect or disconnect from the CORE (assuming their type maps overlap with the message types registered by the client). When the CORE receives a message that matches the set of message types specified during the ``InitMessage`` (or if monitoring is enabled in for inbound messages in the options), it sends a ``NotifyTrafficMessage`` with the peer identity of the sender and the decrypted payload. The same message format (except with ``GNUNET_MESSAGE_TYPE_CORE_NOTIFY_OUTBOUND`` for the message type) is used to notify clients monitoring outbound messages; here, the peer identity given is that of the receiver. .. _Sending: Sending ^^^^^^^ When a client wants to transmit a message, it first requests a transmission slot by sending a ``SendMessageRequest`` which specifies the priority, deadline and size of the message. Note that these values may be ignored by CORE. When CORE is ready for the message, it answers with a ``SendMessageReady`` response. The client can then transmit the payload with a ``SendMessage`` message. Note that the actual message size in the ``SendMessage`` is allowed to be smaller than the size in the original request. A client may at any time send a fresh ``SendMessageRequest``, which then superceeds the previous ``SendMessageRequest``, which is then no longer valid. The client can tell which ``SendMessageRequest`` the CORE service's ``SendMessageReady`` message is for as all of these messages contain a \"unique\" request ID (based on a counter incremented by the client for each request). CORE Peer-to-Peer Protocol .. _The-CORE-Peer_002dto_002dPeer-Protocol: The CORE Peer-to-Peer Protocol ------------------------------ EphemeralKeyMessage creation .. _Creating-the-EphemeralKeyMessage: Creating the EphemeralKeyMessage ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ When the CORE service starts, each peer creates a fresh ephemeral (ECC) public-private key pair and signs the corresponding ``EphemeralKeyMessage`` with its long-term key (which we usually call the peer's identity; the hash of the public long term key is what results in a ``struct GNUNET_PeerIdentity`` in all GNUnet APIs. The ephemeral key is ONLY used for an ECDHE (`Elliptic-curve Diffie---Hellman `__) exchange by the CORE service to establish symmetric session keys. A peer will use the same ``EphemeralKeyMessage`` for all peers for ``REKEY_FREQUENCY``, which is usually 12 hours. After that time, it will create a fresh ephemeral key (forgetting the old one) and broadcast the new ``EphemeralKeyMessage`` to all connected peers, resulting in fresh symmetric session keys. Note that peers independently decide on when to discard ephemeral keys; it is not a protocol violation to discard keys more often. Ephemeral keys are also never stored to disk; restarting a peer will thus always create a fresh ephemeral key. The use of ephemeral keys is what provides `forward secrecy `__. Just before transmission, the ``EphemeralKeyMessage`` is patched to reflect the current sender_status, which specifies the current state of the connection from the point of view of the sender. The possible values are: - ``KX_STATE_DOWN`` Initial value, never used on the network - ``KX_STATE_KEY_SENT`` We sent our ephemeral key, do not know the key of the other peer - ``KX_STATE_KEY_RECEIVED`` This peer has received a valid ephemeral key of the other peer, but we are waiting for the other peer to confirm it's authenticity (ability to decode) via challenge-response. - ``KX_STATE_UP`` The connection is fully up from the point of view of the sender (now performing keep-alive) - ``KX_STATE_REKEY_SENT`` The sender has initiated a rekeying operation; the other peer has so far failed to confirm a working connection using the new ephemeral key .. _Establishing-a-connection: Establishing a connection ^^^^^^^^^^^^^^^^^^^^^^^^^ Peers begin their interaction by sending a ``EphemeralKeyMessage`` to the other peer once the TRANSPORT service notifies the CORE service about the connection. A peer receiving an ``EphemeralKeyMessage`` with a status indicating that the sender does not have the receiver's ephemeral key, the receiver's ``EphemeralKeyMessage`` is sent in response. Additionally, if the receiver has not yet confirmed the authenticity of the sender, it also sends an (encrypted)\ ``PingMessage`` with a challenge (and the identity of the target) to the other peer. Peers receiving a ``PingMessage`` respond with an (encrypted) ``PongMessage`` which includes the challenge. Peers receiving a ``PongMessage`` check the challenge, and if it matches set the connection to ``KX_STATE_UP``. .. _Encryption-and-Decryption: Encryption and Decryption ^^^^^^^^^^^^^^^^^^^^^^^^^ All functions related to the key exchange and encryption/decryption of messages can be found in ``gnunet-service-core_kx.c`` (except for the cryptographic primitives, which are in ``util/crypto*.c``). Given the key material from ECDHE, a Key derivation function (`Key derivation function `__) is used to derive two pairs of encryption and decryption keys for AES-256 and TwoFish, as well as initialization vectors and authentication keys (for HMAC (`HMAC `__)). The HMAC is computed over the encrypted payload. Encrypted messages include an iv_seed and the HMAC in the header. Each encrypted message in the CORE service includes a sequence number and a timestamp in the encrypted payload. The CORE service remembers the largest observed sequence number and a bit-mask which represents which of the previous 32 sequence numbers were already used. Messages with sequence numbers lower than the largest observed sequence number minus 32 are discarded. Messages with a timestamp that is less than ``REKEY_TOLERANCE`` off (5 minutes) are also discarded. This of course means that system clocks need to be reasonably synchronized for peers to be able to communicate. Additionally, as the ephemeral key changes every 12 hours, a peer would not even be able to decrypt messages older than 12 hours. .. _Type-maps: Type maps ^^^^^^^^^ Once an encrypted connection has been established, peers begin to exchange type maps. Type maps are used to allow the CORE service to determine which (encrypted) connections should be shown to which applications. A type map is an array of 65536 bits representing the different types of messages understood by applications using the CORE service. Each CORE service maintains this map, simply by setting the respective bit for each message type supported by any of the applications using the CORE service. Note that bits for message types embedded in higher-level protocols (such as MESH) will not be included in these type maps. Typically, the type map of a peer will be sparse. Thus, the CORE service attempts to compress its type map using ``gzip``-style compression (\"deflate\") prior to transmission. However, if the compression fails to compact the map, the map may also be transmitted without compression (resulting in ``GNUNET_MESSAGE_TYPE_CORE_COMPRESSED_TYPE_MAP`` or ``GNUNET_MESSAGE_TYPE_CORE_BINARY_TYPE_MAP`` messages respectively). Upon receiving a type map, the respective CORE service notifies applications about the connection to the other peer if they support any message type indicated in the type map (or no message type at all). If the CORE service experience a connect or disconnect event from an application, it updates its type map (setting or unsetting the respective bits) and notifies its neighbours about the change. The CORE services of the neighbours then in turn generate connect and disconnect events for the peer that sent the type map for their respective applications. As CORE messages may be lost, the CORE service confirms receiving a type map by sending back a ``GNUNET_MESSAGE_TYPE_CORE_CONFIRM_TYPE_MAP``. If such a confirmation (with the correct hash of the type map) is not received, the sender will retransmit the type map (with exponential back-off).