GNS
The GNS API itself is extremely simple. Clients first connect to the GNS
service using GNUNET_GNS_connect. They can then perform lookups
using GNUNET_GNS_lookup or cancel pending lookups using
GNUNET_GNS_lookup_cancel. Once finished, clients disconnect using
GNUNET_GNS_disconnect.
Looking up records
GNUNET_GNS_lookup takes a number of arguments:
- handle This is simply the GNS connection handle from
GNUNET_GNS_connect.- name The client needs to specify the name to
be resolved. This can be any valid DNS or GNS hostname.
- zone The client
needs to specify the public key of the GNS zone against which the resolution should be done. Note that a key must be provided, the client should look up plausible values using its configuration, the identity service and by attempting to interpret the TLD as a base32-encoded public key.
- type This is the desired GNS or DNS record type
to look for. While all records for the given name will be returned, this can be important if the client wants to resolve record types that themselves delegate resolution, such as CNAME, PKEY or GNS2DNS. Resolving a record of any of these types will only work if the respective record type is specified in the request, as the GNS resolver will otherwise follow the delegation and return the records from the respective destination, instead of the delegating record.
- only_cached This argument should typically be set to
GNUNET_NO. Setting it toGNUNET_YESdisables resolution via the overlay network.- shorten_zone_key If GNS encounters new names during resolution,
their respective zones can automatically be learned and added to the "shorten zone". If this is desired, clients must pass the private key of the shorten zone. If NULL is passed, shortening is disabled.
- proc This argument identifies
the function to call with the result. It is given proc_cls, the number of records found (possibly zero) and the array of the records as arguments. proc will only be called once. After proc,> has been called, the lookup must no longer be canceled.
proc_cls The closure for proc.
Accessing the records
The libgnunetgnsrecord library provides an API to manipulate the GNS
record array that is given to proc. In particular, it offers functions
such as converting record values to human-readable strings (and back).
However, most libgnunetgnsrecord functions are not interesting to
GNS client applications.
For DNS records, the libgnunetdnsparser library provides functions
for parsing (and serializing) common types of DNS records.
Creating records
Creating GNS records is typically done by building the respective record
information (possibly with the help of libgnunetgnsrecord and
libgnunetdnsparser) and then using the libgnunetnamestore to
publish the information. The GNS API is not involved in this operation.
Future work
In the future, we want to expand libgnunetgns to allow applications
to observe shortening operations performed during GNS resolution, for
example so that users can receive visual feedback when this happens.
libgnunetgnsrecord
The libgnunetgnsrecord library is used to manipulate GNS records (in
plaintext or in their encrypted format). Applications mostly interact
with libgnunetgnsrecord by using the functions to convert GNS record
values to strings or vice-versa, or to lookup a GNS record type number
by name (or vice-versa). The library also provides various other
functions that are mostly used internally within GNS, such as converting
keys to names, checking for expiration, encrypting GNS records to GNS
blocks, verifying GNS block signatures and decrypting GNS records from
GNS blocks.
We will now discuss the four commonly used functions of the
API. libgnunetgnsrecord does not perform these operations itself,
but instead uses plugins to perform the operation. GNUnet includes
plugins to support common DNS record types as well as standard GNS
record types.
Value handling
GNUNET_GNSRECORD_value_to_string can be used to convert the (binary)
representation of a GNS record value to a human readable, 0-terminated
UTF-8 string. NULL is returned if the specified record type is not
supported by any available plugin.
GNUNET_GNSRECORD_string_to_value can be used to try to convert a
human readable string to the respective (binary) representation of a GNS
record value.
Type handling
GNUNET_GNSRECORD_typename_to_number can be used to obtain the
numeric value associated with a given typename. For example, given the
typename "A" (for DNS A reocrds), the function will return the number
1. A list of common DNS record types is
here. Note
that not all DNS record types are supported by GNUnet GNSRECORD plugins
at this time.
GNUNET_GNSRECORD_number_to_typename can be used to obtain the
typename associated with a given numeric value. For example, given the
type number 1, the function will return the typename "A".
GNS plugins
Adding a new GNS record type typically involves writing (or extending) a
GNSRECORD plugin. The plugin needs to implement the
gnunet_gnsrecord_plugin.h API which provides basic functions that
are needed by GNSRECORD to convert typenames and values of the
respective record type to strings (and back). These gnsrecord plugins
are typically implemented within their respective subsystems. Examples
for such plugins can be found in the GNSRECORD, GNS and CONVERSATION
subsystems.
The libgnunetgnsrecord library is then used to locate, load and
query the appropriate gnsrecord plugin. Which plugin is appropriate is
determined by the record type (which is just a 32-bit integer). The
libgnunetgnsrecord library loads all block plugins that are
installed at the local peer and forwards the application request to the
plugins. If the record type is not supported by the plugin, it should
simply return an error code.
The central functions of the block APIs (plugin and main library) are the same four functions for converting between values and strings, and typenames and numbers documented in the previous subsection.
The GNS Client-Service Protocol
The GNS client-service protocol consists of two simple messages, the
LOOKUP message and the LOOKUP_RESULT. Each LOOKUP message
contains a unique 32-bit identifier, which will be included in the
corresponding response. Thus, clients can send many lookup requests in
parallel and receive responses out-of-order. A LOOKUP request also
includes the public key of the GNS zone, the desired record type and
fields specifying whether shortening is enabled or networking is
disabled. Finally, the LOOKUP message includes the name to be
resolved.
The response includes the number of records and the records themselves
in the format created by GNUNET_GNSRECORD_records_serialize. They
can thus be deserialized using GNUNET_GNSRECORD_records_deserialize.
Hijacking the DNS-Traffic using gnunet-service-dns
This section documents how the gnunet-service-dns (and the
gnunet-helper-dns) intercepts DNS queries from the local system. This is
merely one method for how we can obtain GNS queries. It is also possible
to change resolv.conf to point to a machine running
gnunet-dns2gns or to modify libc’s name system switch (NSS)
configuration to include a GNS resolution plugin. The method described
in this chapter is more of a last-ditch catch-all approach.
gnunet-service-dns enables intercepting DNS traffic using policy
based routing. We MARK every outgoing DNS-packet if it was not sent by
our application. Using a second routing table in the Linux kernel these
marked packets are then routed through our virtual network interface and
can thus be captured unchanged.
Our application then reads the query and decides how to handle it. If
the query can be addressed via GNS, it is passed to
gnunet-service-gns and resolved internally using GNS. In the future,
a reverse query for an address of the configured virtual network could
be answered with records kept about previous forward queries. Queries
that are not hijacked by some application using the DNS service will be
sent to the original recipient. The answer to the query will always be
sent back through the virtual interface with the original nameserver as
source address.
Network Setup Details
The DNS interceptor adds the following rules to the Linux kernel:
iptables -t mangle -I OUTPUT 1 -p udp --sport $LOCALPORT --dport 53 \
-j ACCEPT iptables -t mangle -I OUTPUT 2 -p udp --dport 53 -j MARK \
--set-mark 3 ip rule add fwmark 3 table2 ip route add default via \
$VIRTUALDNS table2
Todo
FIXME: Rewrite to reflect display which is no longer content by line due to the < 74 characters limit.
Line 1 makes sure that all packets coming from a port our application
opened beforehand ($LOCALPORT) will be routed normally. Line 2 marks
every other packet to a DNS-Server with mark 3 (chosen arbitrarily). The
third line adds a routing policy based on this mark 3 via the routing
table.
Importing DNS Zones into GNS
This section discusses the challenges and problems faced when writing the Ascension tool. It also takes a look at possible improvements in the future.
Consider the following diagram that shows the workflow of Ascension:
Further the interaction between components of GNUnet are shown in the diagram below:
DNS Conversion .. _Conversions-between-DNS-and-GNS:
Conversions between DNS and GNS
The differences between the two name systems lies in the details and is not always transparent. For instance an SRV record is converted to a BOX record which is unique to GNS.
This is done by converting to a BOX record from an existing SRV record:
# SRV
# _service._proto.name. TTL class SRV priority weight port target
_sip._tcp.example.com. 14000 IN SRV 0 0 5060 www.example.com.
# BOX
# TTL BOX flags port protocol recordtype priority weight port target
14000 BOX n 5060 6 33 0 0 5060 www.example.com
Other records that need to undergo such transformation is the MX record type, as well as the SOA record type.
Transformation of a SOA record into GNS works as described in the following example. Very important to note are the rname and mname keys.
# BIND syntax for a clean SOA record
IN SOA master.example.com. hostmaster.example.com. (
2017030300 ; serial
3600 ; refresh
1800 ; retry
604800 ; expire
600 ) ; ttl
# Recordline for adding the record
$ gnunet-namestore -z example.com -a -n -t SOA -V \
rname=master.example.com mname=hostmaster.example.com \
2017030300,3600,1800,604800,600 -e 7200s
The transformation of MX records is done in a simple way.
# mail.example.com. 3600 IN MX 10 mail.example.com.
$ gnunet-namestore -z example.com -n mail -R 3600 MX n 10,mail
Finally, one of the biggest struggling points were the NS records that are found in top level domain zones. The intended behaviour for those is to add GNS2DNS records for those so that gnunet-gns can resolve records for those domains on its own. Those require the values from DNS GLUE records, provided they are within the same zone.
The following two examples show one record with a GLUE record and the other one does not have a GLUE record. This takes place in the ‘com’ TLD.
# ns1.example.com 86400 IN A 127.0.0.1
# example.com 86400 IN NS ns1.example.com.
$ gnunet-namestore -z com -n example -R 86400 GNS2DNS n \
example.com@127.0.0.1
# example.com 86400 IN NS ns1.example.org.
$ gnunet-namestore -z com -n example -R 86400 GNS2DNS n \
example.com@ns1.example.org
As you can see, one of the GNS2DNS records has an IP address listed and the other one a DNS name. For the first one there is a GLUE record to do the translation directly and the second one will issue another DNS query to figure out the IP of ns1.example.org.
A solution was found by creating a hierarchical zone structure in GNS and linking the zones using PKEY records to one another. This allows the resolution of the name servers to work within GNS while not taking control over unwanted zones.
Currently the following record types are supported:
A
AAAA
CNAME
MX
NS
SRV
TXT
This is not due to technical limitations but rather a practical ones. The problem occurs with DNSSEC enabled DNS zones. As records within those zones are signed periodically, and every new signature is an update to the zone, there are many revisions of zones. This results in a problem with bigger zones as there are lots of records that have been signed again but no major changes. Also trying to add records that are unknown that require a different format take time as they cause a CLI call of the namestore. Furthermore certain record types need transformation into a GNS compatible format which, depending on the record type, takes more time.
Further a blacklist was added to drop for instance DNSSEC related records. Also if a record type is neither in the white list nor the blacklist it is considered as a loss of data and a message is shown to the user. This helps with transparency and also with contributing, as the not supported record types can then be added accordingly.
DNS Zone Size
Another very big problem exists with very large zones. When migrating a small zone the delay between adding of records and their expiry is negligible. However when working with big zones that easily have more than a few million records this delay becomes a problem.
Records will start to expire well before the zone has finished migrating. This is usually not a problem but can cause a high CPU load when a peer is restarted and the records have expired.
A good solution has not been found yet. One of the idea that floated around was that the records should be added with the s (shadow) flag to keep the records resolvable even if they expired. However this would introduce the problem of how to detect if a record has been removed from the zone and would require deletion of said record(s).
Another problem that still persists is how to refresh records. Expired records are still displayed when calling gnunet-namestore but do not resolve with gnunet-gns. Zonemaster will sign the expired records again and make sure that the records are still valid. With a recent change this was fixed as gnunet-gns to improve the suffix lookup which allows for a fast lookup even with thousands of local egos.
Currently the pace of adding records in general is around 10 records per second. Crypto is the upper limit for adding of records. The performance of your machine can be tested with the perf_crypto_* tools. There is still a big discrepancy between the pace of Ascension and the theoretical limit.
A performance metric for measuring improvements has not yet been implemented in Ascension.
Performance
The performance when migrating a zone using the Ascension tool is limited by a handful of factors. First of all ascension is written in Python3 and calls the CLI tools of GNUnet. This is comparable to a fork and exec call which costs a few CPU cycles. Furthermore all the records that are added to the same label are signed using the zones private key. This signing operation is very resource heavy and was optimized during development by adding the ‘-R’ (Recordline) option to gnunet-namestore which allows to specify multiple records using the CLI tool. Assuming that in a TLD zone every domain has at least two name servers this halves the amount of signatures needed.
Another improvement that could be made is with the addition of multiple threads or using asynchronous subprocesses when opening the GNUnet CLI tools. This could be implemented by simply creating more workers in the program but performance improvements were not tested.
Ascension was tested using different hardware and database backends. Performance differences between SQLite and postgresql are marginal and almost non existent. What did make a huge impact on record adding performance was the storage medium. On a traditional mechanical hard drive adding of records were slow compared to a solid state disk.
In conclusion there are many bottlenecks still around in the program, namely the single threaded implementation and inefficient, sequential calls of gnunet-namestore. In the future a solution that uses the C API would be cleaner and better.