Developer Handbook

This book is intended to be an introduction for programmers that want to extend the GNUnet framework. GNUnet is more than a simple peer-to-peer application.

For developers, GNUnet is:

• developed by a community that believes in the GNU philosophy

• Free Software (Free as in Freedom), licensed under the GNU Affero General Public License

• A set of standards, including coding conventions and architectural rules

• A set of layered protocols, both specifying the communication between peers as well as the communication between components of a single peer

• A set of libraries with well-defined APIs suitable for writing extensions

In particular, the architecture specifies that a peer consists of many processes communicating via protocols. Processes can be written in almost any language. C, Java and Guile APIs exist for accessing existing services and for writing extensions. It is possible to write extensions in other languages by implementing the necessary IPC protocols.

GNUnet can be extended and improved along many possible dimensions, and anyone interested in Free Software and Freedom-enhancing Networking is welcome to join the effort. This Developer Handbook attempts to provide an initial introduction to some of the key design choices and central components of the system.

This part of the GNUnet documentation is far from complete, and we welcome informed contributions, be it in the form of new chapters, sections or insightful comments.

• Contributing
  • Licenses of contributions
  • Copyright Assignment
  • Contributing to the Reference Manual
  • Contributing testcases
• Style and Workflow
  • Naming conventions
    • Header files
    • Binaries
    • Logging
    • Symbols
    • Tests
    • src subdirectories
  • Coding style
  • Continuous integration
  • Commit messages and developer branches
• Repository Structure
  • src Overview
• libgnunetutil
• System Architecture
  • Internal dependencies
  • Subsystem stability
  • Basic Services
    • STATISTICS — Runtime statistics publication
      • Statistics retrieval
      • Setting statistics and updating them
      • Watches
        • The STATISTICS Client-Service Protocol
      • Statistics retrieval
      • Setting and updating statistics
      • Watching for updates
    • ATS — Automatic transport selection
    • TRANSPORT — Overlay transport management
      • Address validation protocol
      • NAT library
      • Distance-Vector plugin
      • SMTP plugin
      • Why use SMTP for a peer-to-peer transport?
      • How does it work?
      • How do I configure my peer?
      • How do I test if it works?
      • How fast is it?
      • Bluetooth plugin
      • What do I need to use the Bluetooth plugin transport?
      • How does it work2?
      • What possible errors should I be aware of?
      • How do I configure my peer2?
      • How can I test it?
      • The implementation of the Bluetooth transport plugin
        • Linux functionality
        • THE INITIALIZATION
        • THE LOOP
        • Details about the broadcast implementation
        • Pending features
      • WLAN plugin
    • TRANSPORT-NG — Next-generation transport management
      • Design goals of TNG
      • HELLO-NG
      • Priorities and preferences
      • Communicators
    • HOSTLIST — HELLO bootstrapping and gossip
      • HELLOs
      • Overview for the HOSTLIST subsystem
        • Features
        • HOSTLIST - Limitations
      • Interacting with the HOSTLIST daemon
      • Hostlist security address validation
        • The HTTP Server
        • Advertising the URL
      • The HOSTLIST client
        • Bootstrapping
      • Usage
    • PEERINFO — Persistent HELLO storage
      • PEERINFO - Features
      • PEERINFO - Limitations
      • DeveloperPeer Information
      • Startup
      • Managing Information
      • Obtaining Information
      • The PEERINFO Client-Service Protocol
        • Connecting to the PEERINFO Service
        • Adding Information to the PEERINFO Service
        • Obtaining Information from the PEERINFO Service
    • PEERSTORE — Extensible local persistent data storage
      • Functionality
      • Architecture
    • CORE — GNUnet link layer
      • When is a peer "connected"?
      • The CORE Client-Service Protocol
        • Setup2
        • Notifications
        • Sending
      • The CORE Peer-to-Peer Protocol
        • Creating the EphemeralKeyMessage
        • Establishing a connection
        • Encryption and Decryption
        • Type maps
    • NSE — Network size estimation
      • Motivation
        • Example
        • Algorithm
        • Target value
        • Timing
        • Controlled Flooding
        • Calculating the estimate
        • Results
        • Examples
      • The NSE Client-Service Protocol
      • The NSE Peer-to-Peer Protocol
    • DHT — Distributed Hash Table
      • Block library and plugins
        • What is a Block?
        • The API of libgnunetblock
        • Queries
        • Sample Code
        • Conclusion2
        • GET
        • PUT
        • MONITOR
        • DHT Routing Options
      • The DHT Client-Service Protocol
        • PUTting data into the DHT
        • GETting data from the DHT
        • Monitoring the DHT
      • The DHT Peer-to-Peer Protocol
        • Routing GETs or PUTs
        • PUTting data into the DHT
        • GETting data from the DHT
    • REGEX — Service discovery using regular expressions
      • How to run the regex profiler
    • CADET — Confidential Ad-hoc Decentralized End-to-end Transport
    • RPS — Random peer sampling
      • Brahms
  • Peer-to-Peer Set Operations
    • SET — Peer to peer set operations (Deprecated)
      • Local Sets
      • Set Modifications
      • Set Operations
      • Result Elements
        • Sets
        • Listeners
        • Operations
        • Supplying a Set
        • The Result Callback
      • The SET Client-Service Protocol
        • Creating Sets
        • Listeners
        • Initiating Operations
        • Modifying Sets
        • Results and Operation Status
        • Iterating Sets
      • The SET Intersection Peer-to-Peer Protocol
        • The Bloom filter exchange
        • Salt
      • The SET Union Peer-to-Peer Protocol
    • SETI — Peer to peer set intersections
      • Intersection Sets
      • Set Intersection Modifications
      • Set Intersection Operations
      • Intersection Result Elements
        • Intersection Set API
        • Intersection Listeners
        • Intersection Operations
        • Supplying a Set for Intersection
        • The Intersection Result Callback
      • The SETI Client-Service Protocol
        • Creating Intersection Sets
        • Listeners for Intersection
        • Initiating Intersection Operations
        • Modifying Intersection Sets
        • Intersection Results and Operation Status
      • The SETI Intersection Peer-to-Peer Protocol
        • The Bloom filter exchange in SETI
        • Intersection Salt
    • SETU — Peer to peer set unions
      • Union Sets
      • Set Union Modifications
      • Set Union Operations
      • Union Result Elements
        • Union Set API
        • Union Listeners
        • Union Operations
        • Supplying a Set for Union
        • The Union Result Callback
      • The SETU Client-Service Protocol
        • Creating Union Sets
        • Listeners for Union
        • Initiating Union Operations
        • Modifying Union Sets
        • Union Results and Operation Status
          • The SETU Union Peer-to-Peer Protocol
  • VPN and VPN Support
  • GNS and GNS Support
    • GNS — the GNU Name System
      • Looking up records
      • Accessing the records
      • Creating records
      • Future work
        • libgnunetgnsrecord
      • Value handling
      • Type handling
        • GNS plugins
        • The GNS Client-Service Protocol
        • Hijacking the DNS-Traffic using gnunet-service-dns
      • Network Setup Details
        • Importing DNS Zones into GNS
      • Conversions between DNS and GNS
      • DNS Zone Size
      • Performance
        • Registering names using the FCFS daemon
      • Obtaining information from the daemon
      • Submitting data to the daemon
      • Customizing the HTML output
    • NAMECACHE — DHT caching of GNS results
      • The NAMECACHE Client-Service Protocol
        • Lookup
        • Store
      • The NAMECACHE Plugin API
        • Lookup2
        • Store2
    • NAMESTORE — Storage of local GNS zones
      • Editing Zone Information
      • Transactional operations
      • Iterating Zone Information
      • Monitoring Zone Information
  • Applications
    • FS — File sharing over GNUnet
      • Namespace Advertisements
      • KSBlocks
        • File-sharing persistence directory structure
    • IDENTITY — Ego management
      • Connecting to the service
      • Operations on Egos
      • The anonymous Ego
      • Convenience API to lookup a single ego
      • Associating egos with service functions
        • The IDENTITY Client-Service Protocol
    • REVOCATION — Ego key revocation
      • Dissemination
      • Revocation Message Design Requirements
        • Querying for revoked keys
        • Preparing revocations
        • Issuing revocations
      • The REVOCATION Client-Service Protocol
      • The REVOCATION Peer-to-Peer Protocol
    • MESSENGER — Room-based end-to-end messaging
      • MERGE-message
      • INVITE-message
      • TEXT-message
      • FILE-message
      • DELETE-message
        • Member sessions
    • REST — RESTful GNUnet Web APIs
      • Namespace considerations
      • Endpoint documentation
• REST API
  • Configuration
  • Conventions
  • Identity
    • Definition
      • Identity
      • Error Response
      • Response Code
    • Requests
      • GET Requests
      • POST Request
      • PUT Request
      • DELETE Request
      • OPTIONS Request
  • Namestore
    • Data model
    • Requests
  • GNS
    • Error Response
      • Requests
    • GET Request
    • OPTIONS Request
  • Peerinfo
    • Definition
      • Peer
      • Peerinfo Response
      • Error Response
    • Requests
      • GET Request
      • OPTIONS Request
  • Config
    • Definition
      • Configuration
      • Error Response
      • Response Code
    • Requests
      • GET Requests
      • POST Request
      • OPTIONS Request
• C Tutorial
  • Introduction to GNUnet Architecture
  • First Steps with GNUnet
    • Configure your peer
    • Start a peer
    • Monitor a peer
    • Starting Two Peers by Hand
      • Setup a second peer
      • Start the second peer and connect the peers
      • How to connect manually
    • Starting Peers Using the Testbed Service
  • Developing Applications
    • gnunet-ext
    • Adapting the Template
    • Writing a Client Application
      • Handling command-line options
      • Writing a Client Library
        • Connecting to the Service
        • Sending messages
        • Receiving Replies from the Service
      • Writing a user interface
    • Writing a Service
      • Code Placement
      • Starting a Service
    • Interacting directly with other Peers using the CORE Service
      • New P2P connections
      • Receiving P2P Messages
      • Sending P2P Messages
      • End of P2P connections
    • Storing peer-specific data using the PEERSTORE service
      • Storing records
      • Retrieving records
      • Monitoring records
      • Disconnecting from PEERSTORE
    • Using the DHT
      • Storing data in the DHT
      • Obtaining data from the DHT
      • Implementing a block plugin
        • Validating requests and replies
        • Deriving a key from a reply
        • Initialization of the plugin
        • Shutdown of the plugin
        • Integration of the plugin with the build system
      • Monitoring the DHT
    • Debugging with gnunet-arm
  • GNU Free Documentation License
• Doxygen