GNUnet  0.19.3
gnunet-zoneimport.c File Reference

import a DNS zone for publication in GNS, incremental More...

Include dependency graph for gnunet-zoneimport.c:

Go to the source code of this file.

Data Structures

struct  Zone
 Some zones may include authoritative records for other zones, such as foo.com.uk or bar.com.fr. More...
 
struct  Record
 Record for the request to be stored by GNS. More...
 
struct  Request
 Request we should make. More...
 
struct  GlueClosure
 Closure for check_for_glue. More...
 
struct  ProcessRecordContext
 Closure for process_record(). More...
 

Macros

#define THRESH   100
 Maximum number of queries pending at the same time. More...
 
#define TIME_THRESH   10
 TIME_THRESH is in usecs. More...
 
#define MAX_RETRIES   5
 How often do we retry a query before giving up for good? More...
 
#define MAX_SERIES   10
 How many DNS requests do we at most issue in rapid series? More...
 
#define SERIES_DELAY    GNUNET_TIME_relative_multiply (GNUNET_TIME_UNIT_MICROSECONDS, 10)
 How long do we wait at least between series of requests? More...
 
#define NS_BATCH_SIZE   1024
 How many requests do we request from NAMESTORE in one batch during our initial iteration? More...
 

Typedefs

typedef void(* RecordProcessor) (void *cls, const struct GNUNET_DNSPARSER_Record *rec)
 Callback for for_all_records. More...
 

Functions

static void for_all_records (const struct GNUNET_DNSPARSER_Packet *p, RecordProcessor rp, void *rp_cls)
 Call rp for each record in p, regardless of what response section it is in. More...
 
static const char * get_label (struct Request *req)
 Return just the label of the hostname in req. More...
 
static void * build_dns_query (struct Request *req, size_t *raw_size)
 Build DNS query for hostname. More...
 
static void free_records (struct Request *req)
 Free records associated with req. More...
 
static void free_request (struct Request *req)
 Free req and data structures reachable from it. More...
 
static void process_queue (void *cls)
 Process as many requests as possible from the queue. More...
 
static void insert_sorted (struct Request *req)
 Insert req into DLL sorted by next fetch time. More...
 
static void add_record (struct Request *req, uint32_t type, struct GNUNET_TIME_Absolute expiration_time, const void *data, size_t data_len)
 Add record to the GNS record set for req. More...
 
static void check_for_glue (void *cls, const struct GNUNET_DNSPARSER_Record *rec)
 Try to find glue records for a given NS record. More...
 
static void process_record (void *cls, const struct GNUNET_DNSPARSER_Record *rec)
 We received rec for req. More...
 
static void store_completed_cb (void *cls, enum GNUNET_ErrorCode ec)
 
static void process_result (void *cls, const struct GNUNET_TUN_DnsHeader *dns, size_t dns_len)
 Function called with the result of a DNS resolution. More...
 
static int free_request_it (void *cls, const struct GNUNET_HashCode *key, void *value)
 Iterator called during do_shutdown() to free requests in the ns_pending map. More...
 
static void do_shutdown (void *cls)
 Clean up and terminate the process. More...
 
static void iterate_zones (void *cls)
 Iterate over all of the zones we care about and see which records we may need to re-fetch when. More...
 
static void ns_lookup_error_cb (void *cls)
 Function called if GNUNET_NAMESTORE_records_lookup() failed. More...
 
static void ns_lookup_result_cb (void *cls, const struct GNUNET_IDENTITY_PrivateKey *key, const char *label, unsigned int rd_count, const struct GNUNET_GNSRECORD_Data *rd)
 Process a record that was stored in the namestore. More...
 
static void queue (const char *hostname)
 Add hostname to the list of requests to be made. More...
 
static int move_to_queue (void *cls, const struct GNUNET_HashCode *key, void *value)
 We have completed the initial iteration over the namestore's database. More...
 
static void process_stdin (void *cls)
 Begin processing hostnames from stdin. More...
 
static void identity_cb (void *cls, struct GNUNET_IDENTITY_Ego *ego, void **ctx, const char *name)
 Method called to inform about the egos of this peer. More...
 
static void run (void *cls, char *const *args, const char *cfgfile, const struct GNUNET_CONFIGURATION_Handle *cfg)
 Process requests from the queue, then if the queue is not empty, try again. More...
 
int main (int argc, char *const *argv)
 Call with IP address of resolver to query. More...
 

Variables

static struct GNUNET_TIME_Relative minimum_expiration_time
 How long do DNS records have to last at least after being imported? More...
 
static unsigned int map_size = 1024
 Command-line argument specifying desired size of the hash map with all of our pending names. More...
 
static struct GNUNET_IDENTITY_Handleid
 Handle to the identity service. More...
 
static struct GNUNET_NAMESTORE_Handlens
 Namestore handle. More...
 
static struct GNUNET_STATISTICS_Handlestats
 Handle to the statistics service. More...
 
static struct GNUNET_DNSSTUB_Contextctx
 Context for DNS resolution. More...
 
static unsigned int pending
 The number of DNS queries that are outstanding. More...
 
static unsigned int pending_rs
 The number of NAMESTORE record store operations that are outstanding. More...
 
static unsigned int lookups
 Number of lookups we performed overall. More...
 
static unsigned int cached
 Number of records we had cached. More...
 
static unsigned int rejects
 How many hostnames did we reject (malformed). More...
 
static unsigned int failures
 Number of lookups that failed. More...
 
static unsigned int records
 Number of records we found. More...
 
static unsigned int record_sets
 Number of record sets given to namestore. More...
 
static struct GNUNET_CONTAINER_Heapreq_heap
 Heap of all requests to perform, sorted by the time we should next do the request (i.e. More...
 
static struct Requestreq_head
 Active requests are kept in a DLL. More...
 
static struct Requestreq_tail
 Active requests are kept in a DLL. More...
 
static struct GNUNET_SCHEDULER_Taskt
 Main task. More...
 
static struct GNUNET_CONTAINER_MultiHashMapns_pending
 Hash map of requests for which we may still get a response from the namestore. More...
 
static struct GNUNET_NAMESTORE_ZoneIteratorzone_it
 Current zone iteration handle. More...
 
static struct Zonezone_head
 Head of list of zones we are managing. More...
 
static struct Zonezone_tail
 Tail of list of zones we are managing. More...
 
static uint64_t ns_iterator_trigger_next
 After how many more results must ns_lookup_result_cb() ask the namestore for more? More...
 
static uint64_t total_dns_latency_cnt
 Number of DNS requests counted in latency total. More...
 
static struct GNUNET_TIME_Relative total_dns_latency
 Sum of DNS latencies observed. More...
 
static uint64_t total_reg_proc_dns
 Number of records processed (DNS lookup, no NAMESTORE) in total. More...
 
static uint64_t total_reg_proc_dns_ns
 Number of records processed (DNS lookup, with NAMESTORE) in total. More...
 
static struct GNUNET_TIME_Absolute start_time_reg_proc
 Start time of the regular processing. More...
 
static struct GNUNET_TIME_Absolute sleep_time_reg_proc
 Last time we worked before going idle. More...
 
static struct GNUNET_TIME_Relative idle_time
 Time we slept just waiting for work. More...
 

Detailed Description

import a DNS zone for publication in GNS, incremental

Author
Christian Grothoff

Definition in file gnunet-zoneimport.c.

Macro Definition Documentation

◆ THRESH

#define THRESH   100

Maximum number of queries pending at the same time.

Definition at line 36 of file gnunet-zoneimport.c.

◆ TIME_THRESH

#define TIME_THRESH   10

TIME_THRESH is in usecs.

How quickly do we submit fresh queries. Used as an additional throttle.

Definition at line 42 of file gnunet-zoneimport.c.

◆ MAX_RETRIES

#define MAX_RETRIES   5

How often do we retry a query before giving up for good?

Definition at line 47 of file gnunet-zoneimport.c.

◆ MAX_SERIES

#define MAX_SERIES   10

How many DNS requests do we at most issue in rapid series?

Definition at line 52 of file gnunet-zoneimport.c.

◆ SERIES_DELAY

How long do we wait at least between series of requests?

Definition at line 57 of file gnunet-zoneimport.c.

◆ NS_BATCH_SIZE

#define NS_BATCH_SIZE   1024

How many requests do we request from NAMESTORE in one batch during our initial iteration?

Definition at line 69 of file gnunet-zoneimport.c.

Typedef Documentation

◆ RecordProcessor

typedef void(* RecordProcessor) (void *cls, const struct GNUNET_DNSPARSER_Record *rec)

Callback for for_all_records.

Parameters
clsclosure
reca DNS record

Definition at line 366 of file gnunet-zoneimport.c.

Function Documentation

◆ for_all_records()

static void for_all_records ( const struct GNUNET_DNSPARSER_Packet p,
RecordProcessor  rp,
void *  rp_cls 
)
static

Call rp for each record in p, regardless of what response section it is in.

Parameters
ppacket from DNS
rpfunction to call
rp_clsclosure for rp

Definition at line 379 of file gnunet-zoneimport.c.

382 {
383  for (unsigned int i = 0; i < p->num_answers; i++)
384  {
385  struct GNUNET_DNSPARSER_Record *rs = &p->answers[i];
386 
387  rp (rp_cls, rs);
388  }
389  for (unsigned int i = 0; i < p->num_authority_records; i++)
390  {
391  struct GNUNET_DNSPARSER_Record *rs = &p->authority_records[i];
392 
393  rp (rp_cls, rs);
394  }
395  for (unsigned int i = 0; i < p->num_additional_records; i++)
396  {
397  struct GNUNET_DNSPARSER_Record *rs = &p->additional_records[i];
398 
399  rp (rp_cls, rs);
400  }
401 }
static char * rp
Relying party.
static struct GNUNET_OS_Process * p
Helper process we started.
Definition: gnunet-uri.c:38
A DNS response record.

References p, and rp.

Referenced by process_record(), and process_result().

Here is the caller graph for this function:

◆ get_label()

static const char* get_label ( struct Request req)
static

Return just the label of the hostname in req.

Parameters
reqrequest to process hostname of
Returns
statically allocated pointer to the label, overwritten upon the next request!

Definition at line 412 of file gnunet-zoneimport.c.

413 {
414  static char label[64];
415  const char *dot;
416 
417  dot = strchr (req->hostname, (unsigned char) '.');
418  if (NULL == dot)
419  {
420  GNUNET_break (0);
421  return NULL;
422  }
423  if (((size_t) (dot - req->hostname)) >= sizeof(label))
424  {
425  GNUNET_break (0);
426  return NULL;
427  }
428  GNUNET_memcpy (label, req->hostname, dot - req->hostname);
429  label[dot - req->hostname] = '\0';
430  return label;
431 }
#define GNUNET_memcpy(dst, src, n)
Call memcpy() but check for n being 0 first.
#define GNUNET_break(cond)
Use this for internal assertion violations that are not fatal (can be handled) but should not occur.
char * hostname
Hostname we are resolving.

References GNUNET_break, GNUNET_memcpy, and Request::hostname.

Referenced by ns_lookup_result_cb(), and process_result().

Here is the caller graph for this function:

◆ build_dns_query()

static void* build_dns_query ( struct Request req,
size_t *  raw_size 
)
static

Build DNS query for hostname.

Parameters
hostnamehost to build query for
[out]raw_sizenumber of bytes in the query
Returns
NULL on error, otherwise pointer to statically (!) allocated query buffer

Definition at line 443 of file gnunet-zoneimport.c.

444 {
445  static char raw[512];
446  char *rawp;
447  struct GNUNET_DNSPARSER_Packet p;
448  struct GNUNET_DNSPARSER_Query q;
449  int ret;
450 
451  q.name = (char *) req->hostname;
453  q.dns_traffic_class = GNUNET_TUN_DNS_CLASS_INTERNET;
454 
455  memset (&p, 0, sizeof(p));
456  p.num_queries = 1;
457  p.queries = &q;
458  p.id = req->id;
459  ret = GNUNET_DNSPARSER_pack (&p, UINT16_MAX, &rawp, raw_size);
460  if (GNUNET_OK != ret)
461  {
462  if (GNUNET_NO == ret)
463  GNUNET_free (rawp);
465  "Failed to pack query for hostname `%s'\n",
466  req->hostname);
467  rejects++;
468  return NULL;
469  }
470  if (*raw_size > sizeof(raw))
471  {
473  "Failed to pack query for hostname `%s'\n",
474  req->hostname);
475  rejects++;
476  GNUNET_break (0);
477  GNUNET_free (rawp);
478  return NULL;
479  }
480  GNUNET_memcpy (raw, rawp, *raw_size);
481  GNUNET_free (rawp);
482  return raw;
483 }
static int ret
Return value of the commandline.
Definition: gnunet-abd.c:81
static int raw
raw output
Definition: gnunet-gns.c:78
static struct GNUNET_REVOCATION_Query * q
Handle for revocation query.
static unsigned int rejects
How many hostnames did we reject (malformed).
#define GNUNET_DNSPARSER_TYPE_NS
int GNUNET_DNSPARSER_pack(const struct GNUNET_DNSPARSER_Packet *p, uint16_t max, char **buf, size_t *buf_length)
Given a DNS packet p, generate the corresponding UDP payload.
Definition: dnsparser.c:1259
#define GNUNET_log(kind,...)
@ GNUNET_OK
@ GNUNET_NO
@ GNUNET_ERROR_TYPE_ERROR
#define GNUNET_free(ptr)
Wrapper around free.
#define GNUNET_TUN_DNS_CLASS_INTERNET
A few common DNS classes (ok, only one is common, but I list a couple more to make it clear what we'r...
Easy-to-process, parsed version of a DNS packet.
uint16_t id
random 16-bit DNS query identifier.

References GNUNET_break, GNUNET_DNSPARSER_pack(), GNUNET_DNSPARSER_TYPE_NS, GNUNET_ERROR_TYPE_ERROR, GNUNET_free, GNUNET_log, GNUNET_memcpy, GNUNET_NO, GNUNET_OK, GNUNET_TUN_DNS_CLASS_INTERNET, Request::hostname, Request::id, p, q, raw, rejects, and ret.

Referenced by process_queue().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ free_records()

static void free_records ( struct Request req)
static

Free records associated with req.

Parameters
reqrequest to free records of

Definition at line 492 of file gnunet-zoneimport.c.

493 {
494  struct Record *rec;
495 
496  /* Free records */
497  while (NULL != (rec = req->rec_head))
498  {
500  GNUNET_free (rec);
501  }
502 }
#define GNUNET_CONTAINER_DLL_remove(head, tail, element)
Remove an element from a DLL.
Record for the request to be stored by GNS.
struct Record * rec_head
Head of records that should be published in GNS for this hostname.
struct Record * rec_tail
Tail of records that should be published in GNS for this hostname.

References GNUNET_CONTAINER_DLL_remove, GNUNET_free, Request::rec_head, and Request::rec_tail.

Referenced by free_request(), ns_lookup_result_cb(), and store_completed_cb().

Here is the caller graph for this function:

◆ free_request()

static void free_request ( struct Request req)
static

Free req and data structures reachable from it.

Parameters
reqrequest to free

Definition at line 511 of file gnunet-zoneimport.c.

512 {
513  free_records (req);
514  GNUNET_free (req);
515 }
static void free_records(struct Request *req)
Free records associated with req.

References free_records(), and GNUNET_free.

Referenced by do_shutdown(), free_request_it(), process_queue(), and process_result().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ process_queue()

static void process_queue ( void *  cls)
static

Process as many requests as possible from the queue.

Parameters
clsNULL

Definition at line 1171 of file gnunet-zoneimport.c.

1172 {
1173  struct Request *req;
1174  unsigned int series;
1175  void *raw;
1176  size_t raw_size;
1177  struct GNUNET_TIME_Relative delay;
1178 
1179  (void) cls;
1182  series = 0;
1183  t = NULL;
1184  while (pending + pending_rs < THRESH)
1185  {
1187  if (NULL == req)
1188  break;
1189  if (NULL != req->qe)
1190  return; /* namestore op still pending */
1191  if (NULL != req->rs)
1192  {
1193  GNUNET_break (0);
1194  return; /* already submitted */
1195  }
1197  break;
1199  req->hn = NULL;
1201  GNUNET_assert (NULL == req->rs);
1203  "Requesting resolution for `%s'\n",
1204  req->hostname);
1205  raw = build_dns_query (req, &raw_size);
1206  if (NULL == raw)
1207  {
1208  GNUNET_break (0);
1209  free_request (req);
1210  continue;
1211  }
1213  req->rs = GNUNET_DNSSTUB_resolve (ctx, raw, raw_size, &process_result, req);
1214  GNUNET_assert (NULL != req->rs);
1215  req->issue_num++;
1216  lookups++;
1217  pending++;
1218  series++;
1219  if (series > MAX_SERIES)
1220  break;
1221  }
1222  if (pending + pending_rs >= THRESH)
1223  {
1225  "Stopped processing queue (%u+%u/%u)]\n",
1226  pending,
1227  pending_rs,
1228  THRESH);
1229  return; /* wait for replies */
1230  }
1232  if (NULL == req)
1233  {
1235  "Stopped processing queue: empty queue\n");
1236  return;
1237  }
1239  {
1241  "Waiting until %s for next record (`%s') to expire\n",
1243  req->hostname);
1244  if (NULL != t)
1248  return;
1249  }
1250  GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "Throttling\n");
1251  if (NULL != t)
1255 }
static struct GNUNET_TIME_Relative delay
When should dkg communication start?
#define THRESH
Maximum number of queries pending at the same time.
static struct GNUNET_DNSSTUB_Context * ctx
Context for DNS resolution.
static struct GNUNET_CONTAINER_Heap * req_heap
Heap of all requests to perform, sorted by the time we should next do the request (i....
static void free_request(struct Request *req)
Free req and data structures reachable from it.
static void process_queue(void *cls)
Process as many requests as possible from the queue.
static unsigned int pending
The number of DNS queries that are outstanding.
#define MAX_SERIES
How many DNS requests do we at most issue in rapid series?
static struct GNUNET_TIME_Absolute sleep_time_reg_proc
Last time we worked before going idle.
static struct GNUNET_SCHEDULER_Task * t
Main task.
static struct GNUNET_TIME_Relative idle_time
Time we slept just waiting for work.
static unsigned int lookups
Number of lookups we performed overall.
static void * build_dns_query(struct Request *req, size_t *raw_size)
Build DNS query for hostname.
static unsigned int pending_rs
The number of NAMESTORE record store operations that are outstanding.
static void process_result(void *cls, const struct GNUNET_TUN_DnsHeader *dns, size_t dns_len)
Function called with the result of a DNS resolution.
static struct Request * req_tail
Active requests are kept in a DLL.
static struct Request * req_head
Active requests are kept in a DLL.
#define SERIES_DELAY
How long do we wait at least between series of requests?
#define GNUNET_CONTAINER_DLL_insert(head, tail, element)
Insert an element at the head of a DLL.
struct GNUNET_DNSSTUB_RequestSocket * GNUNET_DNSSTUB_resolve(struct GNUNET_DNSSTUB_Context *ctx, const void *request, size_t request_len, GNUNET_DNSSTUB_ResultCallback rc, void *rc_cls)
Perform DNS resolution using our default IP from init.
Definition: dnsstub.c:526
void * GNUNET_CONTAINER_heap_remove_root(struct GNUNET_CONTAINER_Heap *heap)
Remove root of the heap.
void * GNUNET_CONTAINER_heap_peek(const struct GNUNET_CONTAINER_Heap *heap)
Get element stored at the root of heap.
#define GNUNET_assert(cond)
Use this for fatal errors that cannot be handled.
@ GNUNET_ERROR_TYPE_DEBUG
@ GNUNET_ERROR_TYPE_INFO
struct GNUNET_SCHEDULER_Task * GNUNET_SCHEDULER_add_at(struct GNUNET_TIME_Absolute at, GNUNET_SCHEDULER_TaskCallback task, void *task_cls)
Schedule a new task to be run at the specified time.
Definition: scheduler.c:1249
void * GNUNET_SCHEDULER_cancel(struct GNUNET_SCHEDULER_Task *task)
Cancel the task with the specified identifier.
Definition: scheduler.c:975
struct GNUNET_SCHEDULER_Task * GNUNET_SCHEDULER_add_delayed(struct GNUNET_TIME_Relative delay, GNUNET_SCHEDULER_TaskCallback task, void *task_cls)
Schedule a new task to be run with a specified delay.
Definition: scheduler.c:1272
struct GNUNET_TIME_Relative GNUNET_TIME_absolute_get_duration(struct GNUNET_TIME_Absolute whence)
Get the duration of an operation as the difference of the current time and the given start time "henc...
Definition: time.c:436
struct GNUNET_TIME_Relative GNUNET_TIME_absolute_get_remaining(struct GNUNET_TIME_Absolute future)
Given a timestamp in the future, how much time remains until then?
Definition: time.c:405
struct GNUNET_TIME_Absolute GNUNET_TIME_absolute_get(void)
Get the current time.
Definition: time.c:111
const char * GNUNET_STRINGS_absolute_time_to_string(struct GNUNET_TIME_Absolute t)
Like asctime, except for GNUnet time.
Definition: strings.c:616
struct GNUNET_TIME_Relative GNUNET_TIME_relative_add(struct GNUNET_TIME_Relative a1, struct GNUNET_TIME_Relative a2)
Add relative times together.
Definition: time.c:585
Time for relative time used by GNUnet, in microseconds.
uint64_t rel_value_us
The actual value.
Request we should make.
struct GNUNET_TIME_Absolute expires
At what time does the (earliest) of the returned records for this name expire? At this point,...
struct GNUNET_NAMESTORE_QueueEntry * qe
Namestore operation pending for this record.
struct GNUNET_CONTAINER_HeapNode * hn
Requests are kept in a heap while waiting to be resolved.
struct GNUNET_DNSSTUB_RequestSocket * rs
Socket used to make the request, NULL if not active.
int issue_num
How often did we issue this query?
struct GNUNET_TIME_Absolute op_start_time
While we are fetching the record, the value is set to the starting time of the GNS operation.

References build_dns_query(), ctx, delay, Request::expires, free_request(), GNUNET_assert, GNUNET_break, GNUNET_CONTAINER_DLL_insert, GNUNET_CONTAINER_heap_peek(), GNUNET_CONTAINER_heap_remove_root(), GNUNET_DNSSTUB_resolve(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_ERROR_TYPE_INFO, GNUNET_log, GNUNET_SCHEDULER_add_at(), GNUNET_SCHEDULER_add_delayed(), GNUNET_SCHEDULER_cancel(), GNUNET_STRINGS_absolute_time_to_string(), GNUNET_TIME_absolute_get(), GNUNET_TIME_absolute_get_duration(), GNUNET_TIME_absolute_get_remaining(), GNUNET_TIME_relative_add(), Request::hn, Request::hostname, idle_time, Request::issue_num, lookups, MAX_SERIES, Request::op_start_time, pending, pending_rs, process_result(), Request::qe, raw, GNUNET_TIME_Relative::rel_value_us, req_head, req_heap, req_tail, Request::rs, SERIES_DELAY, sleep_time_reg_proc, t, and THRESH.

Referenced by insert_sorted(), process_result(), and store_completed_cb().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ insert_sorted()

static void insert_sorted ( struct Request req)
static

Insert req into DLL sorted by next fetch time.

Parameters
reqrequest to insert into req_heap

Definition at line 533 of file gnunet-zoneimport.c.

534 {
535  req->hn =
538  {
539  if (NULL != t)
543  }
544 }
struct GNUNET_CONTAINER_HeapNode * GNUNET_CONTAINER_heap_insert(struct GNUNET_CONTAINER_Heap *heap, void *element, GNUNET_CONTAINER_HeapCostType cost)
Inserts a new element into the heap.
uint64_t abs_value_us
The actual value.

References GNUNET_TIME_Absolute::abs_value_us, Request::expires, GNUNET_CONTAINER_heap_insert(), GNUNET_CONTAINER_heap_peek(), GNUNET_SCHEDULER_add_at(), GNUNET_SCHEDULER_cancel(), GNUNET_TIME_absolute_get(), Request::hn, process_queue(), req_heap, sleep_time_reg_proc, and t.

Referenced by move_to_queue(), ns_lookup_result_cb(), and process_result().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ add_record()

static void add_record ( struct Request req,
uint32_t  type,
struct GNUNET_TIME_Absolute  expiration_time,
const void *  data,
size_t  data_len 
)
static

Add record to the GNS record set for req.

Parameters
reqthe request to expand GNS record set for
typetype to use
expiration_timewhen should rec expire
dataraw data to store
data_lennumber of bytes in data

Definition at line 557 of file gnunet-zoneimport.c.

562 {
563  struct Record *rec;
564 
565  rec = GNUNET_malloc (sizeof(struct Record) + data_len);
566  rec->grd.data = &rec[1];
567  rec->grd.expiration_time = expiration_time.abs_value_us;
568  rec->grd.data_size = data_len;
569  rec->grd.record_type = type;
571  GNUNET_memcpy (&rec[1], data, data_len);
573 }
uint32_t data
The data value.
@ GNUNET_GNSRECORD_RF_NONE
Entry for no flags / cleared flags.
#define GNUNET_malloc(size)
Wrapper around malloc.
uint32_t record_type
Type of the GNS/DNS record.
const void * data
Binary value stored in the DNS record.
size_t data_size
Number of bytes in data.
enum GNUNET_GNSRECORD_Flags flags
Flags for the record.
uint64_t expiration_time
Expiration time for the DNS record.
struct GNUNET_GNSRECORD_Data grd
GNS record.
enum GNUNET_TESTBED_UnderlayLinkModelType type
the type of this model

References GNUNET_TIME_Absolute::abs_value_us, GNUNET_GNSRECORD_Data::data, data, GNUNET_GNSRECORD_Data::data_size, GNUNET_GNSRECORD_Data::expiration_time, GNUNET_GNSRECORD_Data::flags, GNUNET_CONTAINER_DLL_insert, GNUNET_GNSRECORD_RF_NONE, GNUNET_malloc, GNUNET_memcpy, Record::grd, Request::rec_head, Request::rec_tail, GNUNET_GNSRECORD_Data::record_type, and type.

Referenced by check_for_glue(), ns_lookup_result_cb(), and process_record().

Here is the caller graph for this function:

◆ check_for_glue()

static void check_for_glue ( void *  cls,
const struct GNUNET_DNSPARSER_Record rec 
)
static

Try to find glue records for a given NS record.

Parameters
clsa struct GlueClosure *
recrecord that may contain glue information

Definition at line 605 of file gnunet-zoneimport.c.

606 {
607  struct GlueClosure *gc = cls;
608  char dst[65536];
609  size_t dst_len;
610  size_t off;
611  char ip[INET6_ADDRSTRLEN + 1];
612  socklen_t ip_size = (socklen_t) sizeof(ip);
613  struct GNUNET_TIME_Absolute expiration_time;
614  struct GNUNET_TIME_Relative left;
615 
616  if (0 != strcasecmp (rec->name, gc->ns))
617  return;
618  expiration_time = rec->expiration_time;
619  left = GNUNET_TIME_absolute_get_remaining (expiration_time);
620  if (0 == left.rel_value_us)
621  return; /* ignore expired glue records */
622  /* if expiration window is too short, bump it to configured minimum */
623  if (left.rel_value_us < minimum_expiration_time.rel_value_us)
624  expiration_time =
626  dst_len = sizeof(dst);
627  off = 0;
628  switch (rec->type)
629  {
631  if (sizeof(struct in_addr) != rec->data.raw.data_len)
632  {
633  GNUNET_break (0);
634  return;
635  }
636  if (NULL == inet_ntop (AF_INET, rec->data.raw.data, ip, ip_size))
637  {
638  GNUNET_break (0);
639  return;
640  }
642  dst_len,
643  &off,
644  gc->req->hostname)) &&
645  (GNUNET_OK ==
646  GNUNET_DNSPARSER_builder_add_name (dst, dst_len, &off, ip)))
647  {
648  add_record (gc->req,
650  expiration_time,
651  dst,
652  off);
653  gc->found = GNUNET_YES;
654  }
655  break;
656 
658  if (sizeof(struct in6_addr) != rec->data.raw.data_len)
659  {
660  GNUNET_break (0);
661  return;
662  }
663  if (NULL == inet_ntop (AF_INET6, rec->data.raw.data, ip, ip_size))
664  {
665  GNUNET_break (0);
666  return;
667  }
669  dst_len,
670  &off,
671  gc->req->hostname)) &&
672  (GNUNET_OK ==
673  GNUNET_DNSPARSER_builder_add_name (dst, dst_len, &off, ip)))
674  {
675  add_record (gc->req,
677  expiration_time,
678  dst,
679  off);
680  gc->found = GNUNET_YES;
681  }
682  break;
683 
686  dst_len,
687  &off,
688  gc->req->hostname)) &&
690  dst_len,
691  &off,
692  rec->data.hostname)))
693  {
694  add_record (gc->req,
696  expiration_time,
697  dst,
698  off);
699  gc->found = GNUNET_YES;
700  }
701  break;
702 
703  default:
704  /* useless, do nothing */
705  break;
706  }
707 }
#define GNUNET_GNSRECORD_TYPE_GNS2DNS
Delegation to DNS.
static void add_record(struct Request *req, uint32_t type, struct GNUNET_TIME_Absolute expiration_time, const void *data, size_t data_len)
Add record to the GNS record set for req.
static struct GNUNET_TIME_Relative minimum_expiration_time
How long do DNS records have to last at least after being imported?
int GNUNET_DNSPARSER_builder_add_name(char *dst, size_t dst_len, size_t *off, const char *name)
Add a DNS name to the UDP packet at the given location, converting the name to IDNA notation as neces...
Definition: dnsparser.c:889
#define GNUNET_DNSPARSER_TYPE_A
#define GNUNET_DNSPARSER_TYPE_CNAME
#define GNUNET_DNSPARSER_TYPE_AAAA
@ GNUNET_YES
struct GNUNET_TIME_Absolute GNUNET_TIME_relative_to_absolute(struct GNUNET_TIME_Relative rel)
Convert relative time to an absolute time in the future.
Definition: time.c:316
void * data
Binary record data.
size_t data_len
Number of bytes in data.
char * hostname
For NS, CNAME and PTR records, this is the uncompressed 0-terminated hostname.
uint16_t type
See GNUNET_DNSPARSER_TYPE_*.
struct GNUNET_TIME_Absolute expiration_time
When does the record expire?
union GNUNET_DNSPARSER_Record::@24 data
Payload of the record (which one of these is valid depends on the 'type').
char * name
Name of the record that the query is for (0-terminated).
struct GNUNET_DNSPARSER_RawRecord raw
Raw data for all other types.
Time for absolute times used by GNUnet, in microseconds.
Closure for check_for_glue.
struct Request * req
Overall request we are processing.
int found
Set to GNUNET_YES if glue was found.
const char * ns
NS name we are looking for glue for.

References add_record(), GNUNET_DNSPARSER_RawRecord::data, GNUNET_DNSPARSER_Record::data, GNUNET_DNSPARSER_RawRecord::data_len, GNUNET_DNSPARSER_Record::expiration_time, GlueClosure::found, GNUNET_break, GNUNET_DNSPARSER_builder_add_name(), GNUNET_DNSPARSER_TYPE_A, GNUNET_DNSPARSER_TYPE_AAAA, GNUNET_DNSPARSER_TYPE_CNAME, GNUNET_GNSRECORD_TYPE_GNS2DNS, GNUNET_OK, GNUNET_TIME_absolute_get_remaining(), GNUNET_TIME_relative_to_absolute(), GNUNET_YES, Request::hostname, GNUNET_DNSPARSER_Record::hostname, minimum_expiration_time, GNUNET_DNSPARSER_Record::name, GlueClosure::ns, GNUNET_DNSPARSER_Record::raw, GNUNET_TIME_Relative::rel_value_us, GlueClosure::req, and GNUNET_DNSPARSER_Record::type.

Referenced by process_record().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ process_record()

static void process_record ( void *  cls,
const struct GNUNET_DNSPARSER_Record rec 
)
static

We received rec for req.

Remember the answer.

Parameters
clsa struct ProcessRecordContext
recresponse

Definition at line 735 of file gnunet-zoneimport.c.

736 {
737  struct ProcessRecordContext *prc = cls;
738  struct Request *req = prc->req;
739  char dst[65536];
740  size_t dst_len;
741  size_t off;
742  struct GNUNET_TIME_Absolute expiration_time;
743  struct GNUNET_TIME_Relative left;
744 
745  dst_len = sizeof(dst);
746  off = 0;
747  records++;
748  if (0 != strcasecmp (rec->name, req->hostname))
749  {
750  GNUNET_log (
752  "DNS returned record from zone `%s' of type %u while resolving `%s'\n",
753  rec->name,
754  (unsigned int) rec->type,
755  req->hostname);
756  return; /* does not match hostname, might be glue, but
757  not useful for this pass! */
758  }
759  expiration_time = rec->expiration_time;
760  left = GNUNET_TIME_absolute_get_remaining (expiration_time);
761  if (0 == left.rel_value_us)
762  {
764  "DNS returned expired record for `%s'\n",
765  req->hostname);
767  "# expired records obtained from DNS",
768  1,
769  GNUNET_NO);
770  return; /* record expired */
771  }
772 
774  "DNS returned record that expires at %s for `%s'\n",
775  GNUNET_STRINGS_absolute_time_to_string (expiration_time),
776  req->hostname);
777  /* if expiration window is too short, bump it to configured minimum */
778  if (left.rel_value_us < minimum_expiration_time.rel_value_us)
779  expiration_time =
781  switch (rec->type)
782  {
784  struct GlueClosure gc;
785 
786  /* check for glue */
787  gc.req = req;
788  gc.ns = rec->data.hostname;
789  gc.found = GNUNET_NO;
790  for_all_records (prc->p, &check_for_glue, &gc);
791  if ((GNUNET_NO == gc.found) &&
793  dst_len,
794  &off,
795  req->hostname)) &&
797  dst_len,
798  &off,
799  rec->data.hostname)))
800  {
801  /* FIXME: actually check if this is out-of-bailiwick,
802  and if not request explicit resolution... */
804  "Converted OOB (`%s') NS record for `%s'\n",
805  rec->data.hostname,
806  rec->name);
807  add_record (req,
809  expiration_time,
810  dst,
811  off);
812  }
813  else
814  {
816  "Converted NS record for `%s' using glue\n",
817  rec->name);
818  }
819  break;
820  }
821 
824  dst_len,
825  &off,
826  rec->data.hostname))
827  {
829  "Converting CNAME (`%s') record for `%s'\n",
830  rec->data.hostname,
831  rec->name);
832  add_record (req, rec->type, expiration_time, dst, off);
833  }
834  break;
835 
837  /* No support for DNAME in GNS yet! FIXME: support later! */
839  "FIXME: not supported: %s DNAME %s\n",
840  rec->name,
841  rec->data.hostname);
842  break;
843 
845  if (GNUNET_OK ==
846  GNUNET_DNSPARSER_builder_add_mx (dst, dst_len, &off, rec->data.mx))
847  {
849  "Converting MX (`%s') record for `%s'\n",
850  rec->data.mx->mxhost,
851  rec->name);
852  add_record (req, rec->type, expiration_time, dst, off);
853  }
854  break;
855 
857  if (GNUNET_OK ==
858  GNUNET_DNSPARSER_builder_add_soa (dst, dst_len, &off, rec->data.soa))
859  {
860  /* NOTE: GNS does not really use SOAs */
862  "Converting SOA record for `%s'\n",
863  rec->name);
864  add_record (req, rec->type, expiration_time, dst, off);
865  }
866  break;
867 
869  if (GNUNET_OK ==
870  GNUNET_DNSPARSER_builder_add_srv (dst, dst_len, &off, rec->data.srv))
871  {
873  "Converting SRV record for `%s'\n",
874  rec->name);
875  add_record (req, rec->type, expiration_time, dst, off);
876  }
877  break;
878 
881  dst_len,
882  &off,
883  rec->data.hostname))
884  {
885  /* !?: what does a PTR record do in a regular TLD??? */
887  "Converting PTR record for `%s' (weird)\n",
888  rec->name);
889  add_record (req, rec->type, expiration_time, dst, off);
890  }
891  break;
892 
894  if (GNUNET_OK ==
895  GNUNET_DNSPARSER_builder_add_cert (dst, dst_len, &off, rec->data.cert))
896  {
898  "Converting CERT record for `%s'\n",
899  rec->name);
900  add_record (req, rec->type, expiration_time, dst, off);
901  }
902  break;
903 
904  /* Rest is 'raw' encoded and just needs to be copied IF
905  the hostname matches the requested name; otherwise we
906  simply cannot use it. */
910  default:
912  "Converting record of type %u for `%s'\n",
913  (unsigned int) rec->type,
914  rec->name);
915  add_record (req,
916  rec->type,
917  expiration_time,
918  rec->data.raw.data,
919  rec->data.raw.data_len);
920  break;
921  }
922 }
static void for_all_records(const struct GNUNET_DNSPARSER_Packet *p, RecordProcessor rp, void *rp_cls)
Call rp for each record in p, regardless of what response section it is in.
static unsigned int records
Number of records we found.
static struct GNUNET_STATISTICS_Handle * stats
Handle to the statistics service.
static void check_for_glue(void *cls, const struct GNUNET_DNSPARSER_Record *rec)
Try to find glue records for a given NS record.
int GNUNET_DNSPARSER_builder_add_cert(char *dst, size_t dst_len, size_t *off, const struct GNUNET_DNSPARSER_CertRecord *cert)
Add CERT record to the UDP packet at the given location.
Definition: dnsparser.c:1032
#define GNUNET_DNSPARSER_TYPE_SRV
#define GNUNET_DNSPARSER_TYPE_SOA
#define GNUNET_DNSPARSER_TYPE_CERT
#define GNUNET_DNSPARSER_TYPE_PTR
int GNUNET_DNSPARSER_builder_add_soa(char *dst, size_t dst_len, size_t *off, const struct GNUNET_DNSPARSER_SoaRecord *soa)
Add an SOA record to the UDP packet at the given location.
Definition: dnsparser.c:1079
#define GNUNET_DNSPARSER_TYPE_DNAME
int GNUNET_DNSPARSER_builder_add_mx(char *dst, size_t dst_len, size_t *off, const struct GNUNET_DNSPARSER_MxRecord *mx)
Add an MX record to the UDP packet at the given location.
Definition: dnsparser.c:1003
int GNUNET_DNSPARSER_builder_add_srv(char *dst, size_t dst_len, size_t *off, const struct GNUNET_DNSPARSER_SrvRecord *srv)
Add an SRV record to the UDP packet at the given location.
Definition: dnsparser.c:1120
#define GNUNET_DNSPARSER_TYPE_MX
#define GNUNET_DNSPARSER_TYPE_TXT
@ GNUNET_ERROR_TYPE_WARNING
void GNUNET_STATISTICS_update(struct GNUNET_STATISTICS_Handle *handle, const char *name, int64_t delta, int make_persistent)
Set statistic value for the peer.
char * mxhost
Name of the mail server.
struct GNUNET_DNSPARSER_SoaRecord * soa
SOA data for SOA records.
struct GNUNET_DNSPARSER_SrvRecord * srv
SRV data for SRV records.
struct GNUNET_DNSPARSER_MxRecord * mx
MX data for MX records.
struct GNUNET_DNSPARSER_CertRecord * cert
CERT data for CERT records.
Closure for process_record().
struct Request * req
Request we are processing.
struct GNUNET_DNSPARSER_Packet * p
Answer we got back and are currently parsing, or NULL if not active.

References add_record(), GNUNET_DNSPARSER_Record::cert, check_for_glue(), GNUNET_DNSPARSER_RawRecord::data, GNUNET_DNSPARSER_Record::data, GNUNET_DNSPARSER_RawRecord::data_len, GNUNET_DNSPARSER_Record::expiration_time, for_all_records(), GlueClosure::found, GNUNET_DNSPARSER_builder_add_cert(), GNUNET_DNSPARSER_builder_add_mx(), GNUNET_DNSPARSER_builder_add_name(), GNUNET_DNSPARSER_builder_add_soa(), GNUNET_DNSPARSER_builder_add_srv(), GNUNET_DNSPARSER_TYPE_A, GNUNET_DNSPARSER_TYPE_AAAA, GNUNET_DNSPARSER_TYPE_CERT, GNUNET_DNSPARSER_TYPE_CNAME, GNUNET_DNSPARSER_TYPE_DNAME, GNUNET_DNSPARSER_TYPE_MX, GNUNET_DNSPARSER_TYPE_NS, GNUNET_DNSPARSER_TYPE_PTR, GNUNET_DNSPARSER_TYPE_SOA, GNUNET_DNSPARSER_TYPE_SRV, GNUNET_DNSPARSER_TYPE_TXT, GNUNET_ERROR_TYPE_DEBUG, GNUNET_ERROR_TYPE_WARNING, GNUNET_GNSRECORD_TYPE_GNS2DNS, GNUNET_log, GNUNET_NO, GNUNET_OK, GNUNET_STATISTICS_update(), GNUNET_STRINGS_absolute_time_to_string(), GNUNET_TIME_absolute_get_remaining(), GNUNET_TIME_relative_to_absolute(), Request::hostname, GNUNET_DNSPARSER_Record::hostname, minimum_expiration_time, GNUNET_DNSPARSER_Record::mx, GNUNET_DNSPARSER_MxRecord::mxhost, GNUNET_DNSPARSER_Record::name, GlueClosure::ns, ProcessRecordContext::p, GNUNET_DNSPARSER_Record::raw, records, GNUNET_TIME_Relative::rel_value_us, GlueClosure::req, ProcessRecordContext::req, GNUNET_DNSPARSER_Record::soa, GNUNET_DNSPARSER_Record::srv, stats, and GNUNET_DNSPARSER_Record::type.

Referenced by process_result().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ store_completed_cb()

static void store_completed_cb ( void *  cls,
enum GNUNET_ErrorCode  ec 
)
static

Definition at line 926 of file gnunet-zoneimport.c.

927 {
928  static struct GNUNET_TIME_Absolute last;
929  struct Request *req = cls;
930 
931  req->qe = NULL;
932  if (GNUNET_EC_NONE != ec)
933  {
935  "Failed to store zone data for `%s': %s\n",
936  req->hostname,
938  }
939  else
940  {
942  "Stored records under `%s' (%d)\n",
943  req->hostname,
944  ec);
945  }
946  total_reg_proc_dns_ns++; /* finished regular processing */
947  pending_rs--;
948  free_records (req);
949  /* compute NAMESTORE statistics */
950  {
951  static uint64_t total_ns_latency_cnt;
952  static struct GNUNET_TIME_Relative total_ns_latency;
953  struct GNUNET_TIME_Relative ns_latency;
954 
956  total_ns_latency = GNUNET_TIME_relative_add (total_ns_latency, ns_latency);
957  if (0 == total_ns_latency_cnt)
958  last = GNUNET_TIME_absolute_get ();
959  total_ns_latency_cnt++;
960  if (0 == (total_ns_latency_cnt % 1000))
961  {
963 
965  last = GNUNET_TIME_absolute_get ();
966  fprintf (stderr,
967  "Processed 1000 records in %s\n",
970  "# average NAMESTORE PUT latency (μs)",
971  total_ns_latency.rel_value_us
972  / total_ns_latency_cnt,
973  GNUNET_NO);
974  }
975  }
976  /* compute and publish overall velocity */
977  if (0 == (total_reg_proc_dns_ns % 100))
978  {
979  struct GNUNET_TIME_Relative runtime;
980 
982  runtime = GNUNET_TIME_relative_subtract (runtime, idle_time);
983  runtime =
987  "# Regular processing completed without NAMESTORE",
989  GNUNET_NO);
991  "# Regular processing completed with NAMESTORE PUT",
993  GNUNET_NO);
995  "# average request processing latency (μs)",
996  runtime.rel_value_us,
997  GNUNET_NO);
999  "# total time spent idle (μs)",
1001  GNUNET_NO);
1002  }
1003 
1004  if (NULL == t)
1005  {
1008  }
1009 }
const char * GNUNET_ErrorCode_get_hint(enum GNUNET_ErrorCode ec)
Returns a hint for a given error code.
@ GNUNET_EC_NONE
No error (success).
static struct GNUNET_TIME_Absolute start_time_reg_proc
Start time of the regular processing.
static uint64_t total_reg_proc_dns_ns
Number of records processed (DNS lookup, with NAMESTORE) in total.
static uint64_t total_reg_proc_dns
Number of records processed (DNS lookup, no NAMESTORE) in total.
struct GNUNET_SCHEDULER_Task * GNUNET_SCHEDULER_add_now(GNUNET_SCHEDULER_TaskCallback task, void *task_cls)
Schedule a new task to be run as soon as possible.
Definition: scheduler.c:1299
void GNUNET_STATISTICS_set(struct GNUNET_STATISTICS_Handle *handle, const char *name, uint64_t value, int make_persistent)
Set statistic value for the peer.
struct GNUNET_TIME_Relative GNUNET_TIME_relative_subtract(struct GNUNET_TIME_Relative a1, struct GNUNET_TIME_Relative a2)
Subtract relative timestamp from the other.
Definition: time.c:603
struct GNUNET_TIME_Relative GNUNET_TIME_relative_divide(struct GNUNET_TIME_Relative rel, unsigned long long factor)
Divide relative time by a given factor.
Definition: time.c:550
const char * GNUNET_STRINGS_relative_time_to_string(struct GNUNET_TIME_Relative delta, int do_round)
Give relative time in human-readable fancy format.
Definition: strings.c:569
static struct GNUNET_TIME_Relative delta
Definition: speedup.c:36

References delta, free_records(), GNUNET_EC_NONE, GNUNET_ERROR_TYPE_DEBUG, GNUNET_ERROR_TYPE_ERROR, GNUNET_ErrorCode_get_hint(), GNUNET_log, GNUNET_NO, GNUNET_SCHEDULER_add_now(), GNUNET_STATISTICS_set(), GNUNET_STRINGS_relative_time_to_string(), GNUNET_TIME_absolute_get(), GNUNET_TIME_absolute_get_duration(), GNUNET_TIME_relative_add(), GNUNET_TIME_relative_divide(), GNUNET_TIME_relative_subtract(), GNUNET_YES, Request::hostname, idle_time, Request::op_start_time, pending_rs, process_queue(), Request::qe, GNUNET_TIME_Relative::rel_value_us, sleep_time_reg_proc, start_time_reg_proc, stats, t, total_reg_proc_dns, and total_reg_proc_dns_ns.

Referenced by process_result().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ process_result()

static void process_result ( void *  cls,
const struct GNUNET_TUN_DnsHeader dns,
size_t  dns_len 
)
static

Function called with the result of a DNS resolution.

Parameters
clsclosure with the struct Request
dnsdns response, never NULL
dns_lennumber of bytes in dns

Definition at line 1020 of file gnunet-zoneimport.c.

1023 {
1024  struct Request *req = cls;
1025  struct Record *rec;
1026  struct GNUNET_DNSPARSER_Packet *p;
1027  unsigned int rd_count;
1028 
1029  GNUNET_assert (NULL == req->hn);
1030  if (NULL == dns)
1031  {
1032  /* stub gave up */
1034  pending--;
1035  if (NULL == t)
1036  {
1039  }
1041  "Stub gave up on DNS reply for `%s'\n",
1042  req->hostname);
1043  GNUNET_STATISTICS_update (stats, "# DNS lookups timed out", 1, GNUNET_NO);
1044  if (req->issue_num > MAX_RETRIES)
1045  {
1046  failures++;
1047  free_request (req);
1048  GNUNET_STATISTICS_update (stats, "# requests given up on", 1, GNUNET_NO);
1049  return;
1050  }
1052  req->rs = NULL;
1053  insert_sorted (req);
1054  return;
1055  }
1056  if (req->id != dns->id)
1057  {
1059  "DNS ID did not match request, ignoring reply\n");
1060  GNUNET_STATISTICS_update (stats, "# DNS ID mismatches", 1, GNUNET_NO);
1061  return;
1062  }
1065  req->rs = NULL;
1066  pending--;
1067  p = GNUNET_DNSPARSER_parse ((const char *) dns, dns_len);
1068  if (NULL == p)
1069  {
1071  "Failed to parse DNS reply for `%s'\n",
1072  req->hostname);
1073  GNUNET_STATISTICS_update (stats, "# DNS parser errors", 1, GNUNET_NO);
1074  if (NULL == t)
1075  {
1078  }
1079  if (req->issue_num > MAX_RETRIES)
1080  {
1081  failures++;
1082  free_request (req);
1083  GNUNET_STATISTICS_update (stats, "# requests given up on", 1, GNUNET_NO);
1084  return;
1085  }
1086  insert_sorted (req);
1087  return;
1088  }
1089  /* import new records */
1090  req->issue_num = 0; /* success, reset counter! */
1091  {
1092  struct ProcessRecordContext prc = { .req = req, .p = p };
1093 
1094  for_all_records (p, &process_record, &prc);
1095  }
1097  /* count records found, determine minimum expiration time */
1099  {
1100  struct GNUNET_TIME_Relative dns_latency;
1101 
1102  dns_latency = GNUNET_TIME_absolute_get_duration (req->op_start_time);
1106  if (0 == (total_dns_latency_cnt % 1000))
1107  {
1109  "# average DNS lookup latency (μs)",
1112  GNUNET_NO);
1113  }
1114  }
1115  rd_count = 0;
1116  for (rec = req->rec_head; NULL != rec; rec = rec->next)
1117  {
1118  struct GNUNET_TIME_Absolute at;
1119 
1120  at.abs_value_us = rec->grd.expiration_time;
1121  req->expires = GNUNET_TIME_absolute_min (req->expires, at);
1122  rd_count++;
1123  }
1125  "Obtained %u records for `%s'\n",
1126  rd_count,
1127  req->hostname);
1128  /* Instead of going for SOA, simplified for now to look each
1129  day in case we got an empty response */
1130  if (0 == rd_count)
1131  {
1134  "# empty DNS replies (usually NXDOMAIN)",
1135  1,
1136  GNUNET_NO);
1137  }
1138  else
1139  {
1140  record_sets++;
1141  }
1142  /* convert records to namestore import format */
1143  {
1145  unsigned int off = 0;
1146 
1147  /* convert linked list into array */
1148  for (rec = req->rec_head; NULL != rec; rec = rec->next)
1149  rd[off++] = rec->grd;
1150  pending_rs++;
1153  &req->zone->key,
1154  get_label (req),
1155  rd_count,
1156  rd,
1158  req);
1159  GNUNET_assert (NULL != req->qe);
1160  }
1161  insert_sorted (req);
1162 }
static unsigned int rd_count
Number of records for currently parsed set.
static struct GNUNET_GNSRECORD_Data rd[50]
The record data under a single label.
static const char * get_label(struct Request *req)
Return just the label of the hostname in req.
static void store_completed_cb(void *cls, enum GNUNET_ErrorCode ec)
static struct GNUNET_TIME_Relative total_dns_latency
Sum of DNS latencies observed.
static uint64_t total_dns_latency_cnt
Number of DNS requests counted in latency total.
static void insert_sorted(struct Request *req)
Insert req into DLL sorted by next fetch time.
static void process_record(void *cls, const struct GNUNET_DNSPARSER_Record *rec)
We received rec for req.
static struct GNUNET_NAMESTORE_Handle * ns
Namestore handle.
static unsigned int record_sets
Number of record sets given to namestore.
#define MAX_RETRIES
How often do we retry a query before giving up for good?
static unsigned int failures
Number of lookups that failed.
void GNUNET_DNSPARSER_free_packet(struct GNUNET_DNSPARSER_Packet *p)
Free memory taken by a packet.
Definition: dnsparser.c:854
struct GNUNET_DNSPARSER_Packet * GNUNET_DNSPARSER_parse(const char *udp_payload, size_t udp_payload_length)
Parse a UDP payload of a DNS packet in to a nice struct for further processing and manipulation.
Definition: dnsparser.c:656
void GNUNET_DNSSTUB_resolve_cancel(struct GNUNET_DNSSTUB_RequestSocket *rs)
Cancel DNS resolution.
Definition: dnsstub.c:562
#define GNUNET_NZL(l)
Macro used to avoid using 0 for the length of a variable-size array (Non-Zero-Length).
struct GNUNET_NAMESTORE_QueueEntry * GNUNET_NAMESTORE_records_store(struct GNUNET_NAMESTORE_Handle *h, const struct GNUNET_IDENTITY_PrivateKey *pkey, const char *label, unsigned int rd_count, const struct GNUNET_GNSRECORD_Data *rd, GNUNET_NAMESTORE_ContinuationWithStatus cont, void *cont_cls)
Store an item in the namestore.
#define GNUNET_TIME_UNIT_DAYS
One day.
struct GNUNET_TIME_Absolute GNUNET_TIME_absolute_min(struct GNUNET_TIME_Absolute t1, struct GNUNET_TIME_Absolute t2)
Return the minimum of two absolute time values.
Definition: time.c:359
#define GNUNET_TIME_UNIT_FOREVER_ABS
Constant used to specify "forever".
uint16_t id
Unique identifier for the request/response.
struct Record * next
Kept in a DLL.
const struct Zone * zone
Zone responsible for this request.
struct GNUNET_IDENTITY_PrivateKey key
Private key of the zone.

References GNUNET_TIME_Absolute::abs_value_us, GNUNET_GNSRECORD_Data::expiration_time, Request::expires, failures, for_all_records(), free_request(), get_label(), GNUNET_assert, GNUNET_CONTAINER_DLL_remove, GNUNET_DNSPARSER_free_packet(), GNUNET_DNSPARSER_parse(), GNUNET_DNSSTUB_resolve_cancel(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_ERROR_TYPE_ERROR, GNUNET_ERROR_TYPE_INFO, GNUNET_log, GNUNET_NAMESTORE_records_store(), GNUNET_NO, GNUNET_NZL, GNUNET_SCHEDULER_add_now(), GNUNET_STATISTICS_set(), GNUNET_STATISTICS_update(), GNUNET_TIME_absolute_get(), GNUNET_TIME_absolute_get_duration(), GNUNET_TIME_absolute_min(), GNUNET_TIME_relative_add(), GNUNET_TIME_relative_to_absolute(), GNUNET_TIME_UNIT_DAYS, GNUNET_TIME_UNIT_FOREVER_ABS, Record::grd, Request::hn, Request::hostname, Request::id, GNUNET_TUN_DnsHeader::id, insert_sorted(), Request::issue_num, Zone::key, MAX_RETRIES, Record::next, ns, Request::op_start_time, p, pending, pending_rs, process_queue(), process_record(), Request::qe, rd, rd_count, Request::rec_head, record_sets, GNUNET_TIME_Relative::rel_value_us, ProcessRecordContext::req, req_head, req_tail, Request::rs, sleep_time_reg_proc, stats, store_completed_cb(), t, total_dns_latency, total_dns_latency_cnt, total_reg_proc_dns, and Request::zone.

Referenced by process_queue().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ free_request_it()

static int free_request_it ( void *  cls,
const struct GNUNET_HashCode key,
void *  value 
)
static

Iterator called during do_shutdown() to free requests in the ns_pending map.

Parameters
clsNULL
keyunused
valuethe struct Request to free
Returns
GNUNET_OK

Definition at line 1268 of file gnunet-zoneimport.c.

1269 {
1270  struct Request *req = value;
1271 
1272  (void) cls;
1273  (void) key;
1274  free_request (req);
1275  return GNUNET_OK;
1276 }
struct GNUNET_HashCode key
The key used in the DHT.
static char * value
Value of the record to add/remove.

References free_request(), GNUNET_OK, key, and value.

Referenced by do_shutdown().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ do_shutdown()

static void do_shutdown ( void *  cls)
static

Clean up and terminate the process.

Parameters
clsNULL

Definition at line 1285 of file gnunet-zoneimport.c.

1286 {
1287  struct Request *req;
1288  struct Zone *zone;
1289 
1290  (void) cls;
1291  if (NULL != id)
1292  {
1294  id = NULL;
1295  }
1296  if (NULL != t)
1297  {
1299  t = NULL;
1300  }
1301  while (NULL != (req = req_head))
1302  {
1304  if (NULL != req->qe)
1305  GNUNET_NAMESTORE_cancel (req->qe);
1306  free_request (req);
1307  }
1308  while (NULL != (req = GNUNET_CONTAINER_heap_remove_root (req_heap)))
1309  {
1310  req->hn = NULL;
1311  if (NULL != req->qe)
1312  GNUNET_NAMESTORE_cancel (req->qe);
1313  free_request (req);
1314  }
1315  if (NULL != zone_it)
1316  {
1318  zone_it = NULL;
1319  }
1320  if (NULL != ns)
1321  {
1323  ns = NULL;
1324  }
1325  if (NULL != ctx)
1326  {
1328  ctx = NULL;
1329  }
1330  if (NULL != req_heap)
1331  {
1333  req_heap = NULL;
1334  }
1335  if (NULL != ns_pending)
1336  {
1339  ns_pending = NULL;
1340  }
1341  while (NULL != (zone = zone_head))
1342  {
1344  GNUNET_free (zone->domain);
1345  GNUNET_free (zone);
1346  }
1347  if (NULL != stats)
1348  {
1350  stats = NULL;
1351  }
1352 }
static char * zone
Name of the zone being managed.
static struct Zone * zone_tail
Tail of list of zones we are managing.
static int free_request_it(void *cls, const struct GNUNET_HashCode *key, void *value)
Iterator called during do_shutdown() to free requests in the ns_pending map.
static struct GNUNET_NAMESTORE_ZoneIterator * zone_it
Current zone iteration handle.
static struct Zone * zone_head
Head of list of zones we are managing.
static struct GNUNET_CONTAINER_MultiHashMap * ns_pending
Hash map of requests for which we may still get a response from the namestore.
void GNUNET_DNSSTUB_stop(struct GNUNET_DNSSTUB_Context *ctx)
Cleanup DNSSTUB resolver.
Definition: dnsstub.c:705
int GNUNET_CONTAINER_multihashmap_iterate(struct GNUNET_CONTAINER_MultiHashMap *map, GNUNET_CONTAINER_MultiHashMapIteratorCallback it, void *it_cls)
Iterate over all entries in the map.
void GNUNET_CONTAINER_multihashmap_destroy(struct GNUNET_CONTAINER_MultiHashMap *map)
Destroy a hash map.
void GNUNET_CONTAINER_heap_destroy(struct GNUNET_CONTAINER_Heap *heap)
Destroys the heap.
void GNUNET_IDENTITY_disconnect(struct GNUNET_IDENTITY_Handle *h)
Disconnect from identity service.
Definition: identity_api.c:757
void GNUNET_NAMESTORE_disconnect(struct GNUNET_NAMESTORE_Handle *h)
Disconnect from the namestore service (and free associated resources).
void GNUNET_NAMESTORE_cancel(struct GNUNET_NAMESTORE_QueueEntry *qe)
Cancel a namestore operation.
void GNUNET_NAMESTORE_zone_iteration_stop(struct GNUNET_NAMESTORE_ZoneIterator *it)
Stops iteration and releases the namestore handle for further calls.
void GNUNET_STATISTICS_destroy(struct GNUNET_STATISTICS_Handle *h, int sync_first)
Destroy a handle (free all state associated with it).
Some zones may include authoritative records for other zones, such as foo.com.uk or bar....

References ctx, free_request(), free_request_it(), GNUNET_CONTAINER_DLL_remove, GNUNET_CONTAINER_heap_destroy(), GNUNET_CONTAINER_heap_remove_root(), GNUNET_CONTAINER_multihashmap_destroy(), GNUNET_CONTAINER_multihashmap_iterate(), GNUNET_DNSSTUB_stop(), GNUNET_free, GNUNET_IDENTITY_disconnect(), GNUNET_NAMESTORE_cancel(), GNUNET_NAMESTORE_disconnect(), GNUNET_NAMESTORE_zone_iteration_stop(), GNUNET_NO, GNUNET_SCHEDULER_cancel(), GNUNET_STATISTICS_destroy(), Request::hn, ns, ns_pending, Request::qe, req_head, req_heap, req_tail, stats, t, zone, zone_head, zone_it, and zone_tail.

Referenced by run().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ iterate_zones()

static void iterate_zones ( void *  cls)
static

Iterate over all of the zones we care about and see which records we may need to re-fetch when.

Parameters
clsNULL

Definition at line 1585 of file gnunet-zoneimport.c.

1586 {
1587  static struct Zone *last;
1588 
1589  (void) cls;
1590  if (NULL != zone_it)
1591  {
1592  zone_it = NULL;
1594  "Finished iteration over zone `%s'!\n",
1595  last->domain);
1596  /* subtract left-overs from previous iteration */
1598  "# NAMESTORE records requested from cache",
1599  (long long) (-ns_iterator_trigger_next),
1600  GNUNET_NO);
1602  }
1603  GNUNET_assert (NULL != zone_tail);
1604  if (zone_tail == last)
1605  {
1606  /* Done iterating over relevant zones in NAMESTORE, move
1607  rest of hash map to work queue as well. */
1609  "Finished all NAMESTORE iterations!\n");
1611  "# Domain names without cached reply",
1613  GNUNET_NO);
1616  ns_pending = NULL;
1618  total_reg_proc_dns = 0;
1620  return;
1621  }
1622  if (NULL == last)
1623  last = zone_head;
1624  else
1625  last = last->next;
1627  "Starting iteration over zone `%s'!\n",
1628  last->domain);
1629  /* subtract left-overs from previous iteration */
1631  "# NAMESTORE records requested from cache",
1632  1,
1633  GNUNET_NO);
1635  GNUNET_STATISTICS_update (stats, "# zones iterated", 1, GNUNET_NO);
1637  &last->key,
1639  NULL,
1641  last,
1642  &iterate_zones,
1643  NULL);
1644 }
static int move_to_queue(void *cls, const struct GNUNET_HashCode *key, void *value)
We have completed the initial iteration over the namestore's database.
static void ns_lookup_result_cb(void *cls, const struct GNUNET_IDENTITY_PrivateKey *key, const char *label, unsigned int rd_count, const struct GNUNET_GNSRECORD_Data *rd)
Process a record that was stored in the namestore.
static uint64_t ns_iterator_trigger_next
After how many more results must ns_lookup_result_cb() ask the namestore for more?
static void iterate_zones(void *cls)
Iterate over all of the zones we care about and see which records we may need to re-fetch when.
static void ns_lookup_error_cb(void *cls)
Function called if GNUNET_NAMESTORE_records_lookup() failed.
unsigned int GNUNET_CONTAINER_multihashmap_size(const struct GNUNET_CONTAINER_MultiHashMap *map)
Get the number of key-value pairs in the map.
struct GNUNET_NAMESTORE_ZoneIterator * GNUNET_NAMESTORE_zone_iteration_start(struct GNUNET_NAMESTORE_Handle *h, const struct GNUNET_IDENTITY_PrivateKey *zone, GNUNET_SCHEDULER_TaskCallback error_cb, void *error_cb_cls, GNUNET_NAMESTORE_RecordMonitor proc, void *proc_cls, GNUNET_SCHEDULER_TaskCallback finish_cb, void *finish_cb_cls)
char * domain
Domain of the zone (i.e.
struct Zone * next
Kept in a DLL.

References Zone::domain, GNUNET_assert, GNUNET_CONTAINER_multihashmap_destroy(), GNUNET_CONTAINER_multihashmap_iterate(), GNUNET_CONTAINER_multihashmap_size(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_log, GNUNET_NAMESTORE_zone_iteration_start(), GNUNET_NO, GNUNET_STATISTICS_set(), GNUNET_STATISTICS_update(), GNUNET_TIME_absolute_get(), Zone::key, move_to_queue(), Zone::next, ns, ns_iterator_trigger_next, ns_lookup_error_cb(), ns_lookup_result_cb(), ns_pending, start_time_reg_proc, stats, total_reg_proc_dns, total_reg_proc_dns_ns, zone_head, zone_it, and zone_tail.

Referenced by ns_lookup_error_cb(), and process_stdin().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ ns_lookup_error_cb()

static void ns_lookup_error_cb ( void *  cls)
static

Function called if GNUNET_NAMESTORE_records_lookup() failed.

Just logs an error.

Parameters
clsa struct Zone

Definition at line 1372 of file gnunet-zoneimport.c.

1373 {
1374  struct Zone *zone = cls;
1375 
1377  "Failed to load data from namestore for zone `%s'\n",
1378  zone->domain);
1379  zone_it = NULL;
1381  iterate_zones (NULL);
1382 }

References GNUNET_ERROR_TYPE_INFO, GNUNET_log, iterate_zones(), ns_iterator_trigger_next, zone, and zone_it.

Referenced by iterate_zones().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ ns_lookup_result_cb()

static void ns_lookup_result_cb ( void *  cls,
const struct GNUNET_IDENTITY_PrivateKey key,
const char *  label,
unsigned int  rd_count,
const struct GNUNET_GNSRECORD_Data rd 
)
static

Process a record that was stored in the namestore.

Parameters
clsa struct Zone *
keyprivate key of the zone
labellabel of the records
rd_countnumber of entries in rd array, 0 if label was deleted
rdarray of records with data to store

Definition at line 1395 of file gnunet-zoneimport.c.

1400 {
1401  struct Zone *zone = cls;
1402  struct Request *req;
1403  struct GNUNET_HashCode hc;
1404  char *fqdn;
1405 
1408  "Obtained NAMESTORE reply, %llu left in round\n",
1409  (unsigned long long) ns_iterator_trigger_next);
1410  if (0 == ns_iterator_trigger_next)
1411  {
1414  "# NAMESTORE records requested from cache",
1416  GNUNET_NO);
1418  }
1419  GNUNET_asprintf (&fqdn, "%s.%s", label, zone->domain);
1420  GNUNET_CRYPTO_hash (fqdn, strlen (fqdn) + 1, &hc);
1421  GNUNET_free (fqdn);
1423  if (NULL == req)
1424  {
1426  "Ignoring record `%s' in zone `%s': not on my list!\n",
1427  label,
1428  zone->domain);
1429  return;
1430  }
1433  GNUNET_break (0 == GNUNET_memcmp (key, &req->zone->key));
1434  GNUNET_break (0 == strcasecmp (label, get_label (req)));
1435  for (unsigned int i = 0; i < rd_count; i++)
1436  {
1437  struct GNUNET_TIME_Absolute at;
1438 
1440  {
1441  struct GNUNET_TIME_Relative rel;
1442 
1445  }
1446  else
1447  {
1448  at.abs_value_us = rd->expiration_time;
1449  }
1450  add_record (req, rd->record_type, at, rd->data, rd->data_size);
1451  }
1452  if (0 == rd_count)
1453  {
1455  "Empty record set in namestore for `%s'\n",
1456  req->hostname);
1457  }
1458  else
1459  {
1460  unsigned int pos = 0;
1461 
1462  cached++;
1464  for (struct Record *rec = req->rec_head; NULL != rec; rec = rec->next)
1465  {
1466  struct GNUNET_TIME_Absolute at;
1467 
1468  at.abs_value_us = rec->grd.expiration_time;
1469  req->expires = GNUNET_TIME_absolute_min (req->expires, at);
1470  pos++;
1471  }
1472  if (0 == pos)
1475  "Hot-start with %u existing records for `%s'\n",
1476  pos,
1477  req->hostname);
1478  }
1479  free_records (req);
1480 
1482  "Adding `%s' to worklist to start at %s\n",
1483  req->hostname,
1485  insert_sorted (req);
1486 }
#define NS_BATCH_SIZE
How many requests do we request from NAMESTORE in one batch during our initial iteration?
static unsigned int cached
Number of records we had cached.
@ GNUNET_GNSRECORD_RF_RELATIVE_EXPIRATION
This expiration time of the record is a relative time (not an absolute time).
void GNUNET_CRYPTO_hash(const void *block, size_t size, struct GNUNET_HashCode *ret)
Compute hash of a given block.
Definition: crypto_hash.c:41
enum GNUNET_GenericReturnValue GNUNET_CONTAINER_multihashmap_remove(struct GNUNET_CONTAINER_MultiHashMap *map, const struct GNUNET_HashCode *key, const void *value)
Remove the given key-value pair from the map.
void * GNUNET_CONTAINER_multihashmap_get(const struct GNUNET_CONTAINER_MultiHashMap *map, const struct GNUNET_HashCode *key)
Given a key find a value in the map matching the key.
#define GNUNET_memcmp(a, b)
Compare memory in a and b, where both must be of the same pointer type.
int int GNUNET_asprintf(char **buf, const char *format,...) __attribute__((format(printf
Like asprintf, just portable.
void GNUNET_NAMESTORE_zone_iterator_next(struct GNUNET_NAMESTORE_ZoneIterator *it, uint64_t limit)
Calls the record processor specified in GNUNET_NAMESTORE_zone_iteration_start for the next record.
#define GNUNET_TIME_UNIT_ZERO_ABS
Absolute time zero.
A 512-bit hashcode.

References GNUNET_TIME_Absolute::abs_value_us, add_record(), cached, GNUNET_GNSRECORD_Data::data, GNUNET_GNSRECORD_Data::data_size, GNUNET_GNSRECORD_Data::expiration_time, Request::expires, GNUNET_GNSRECORD_Data::flags, free_records(), get_label(), GNUNET_asprintf(), GNUNET_assert, GNUNET_break, GNUNET_CONTAINER_multihashmap_get(), GNUNET_CONTAINER_multihashmap_remove(), GNUNET_CRYPTO_hash(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_ERROR_TYPE_INFO, GNUNET_free, GNUNET_GNSRECORD_RF_RELATIVE_EXPIRATION, GNUNET_log, GNUNET_memcmp, GNUNET_NAMESTORE_zone_iterator_next(), GNUNET_NO, GNUNET_OK, GNUNET_STATISTICS_update(), GNUNET_STRINGS_absolute_time_to_string(), GNUNET_TIME_absolute_min(), GNUNET_TIME_relative_to_absolute(), GNUNET_TIME_UNIT_FOREVER_ABS, GNUNET_TIME_UNIT_ZERO_ABS, Request::hostname, insert_sorted(), key, Zone::key, Record::next, NS_BATCH_SIZE, ns_iterator_trigger_next, ns_pending, rd, rd_count, Request::rec_head, GNUNET_GNSRECORD_Data::record_type, GNUNET_TIME_Relative::rel_value_us, stats, zone, Request::zone, and zone_it.

Referenced by iterate_zones().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ queue()

static void queue ( const char *  hostname)
static

Add hostname to the list of requests to be made.

Parameters
hostnamename to resolve

Definition at line 1495 of file gnunet-zoneimport.c.

1496 {
1497  struct Request *req;
1498  const char *dot;
1499  struct Zone *zone;
1500  size_t hlen;
1501  struct GNUNET_HashCode hc;
1502 
1504  {
1506  "Refusing invalid hostname `%s'\n",
1507  hostname);
1508  rejects++;
1509  return;
1510  }
1511  dot = strchr (hostname, (unsigned char) '.');
1512  if (NULL == dot)
1513  {
1515  "Refusing invalid hostname `%s' (lacks '.')\n",
1516  hostname);
1517  rejects++;
1518  return;
1519  }
1520  for (zone = zone_head; NULL != zone; zone = zone->next)
1521  if (0 == strcmp (zone->domain, dot + 1))
1522  break;
1523  if (NULL == zone)
1524  {
1525  rejects++;
1527  "Domain name `%s' not in ego list!\n",
1528  dot + 1);
1529  return;
1530  }
1531 
1532  hlen = strlen (hostname) + 1;
1533  req = GNUNET_malloc (sizeof(struct Request) + hlen);
1534  req->zone = zone;
1535  req->hostname = (char *) &req[1];
1536  GNUNET_memcpy (req->hostname, hostname, hlen);
1538  UINT16_MAX);
1539  GNUNET_CRYPTO_hash (req->hostname, hlen, &hc);
1541  ns_pending,
1542  &hc,
1543  req,
1545  {
1547  "Duplicate hostname `%s' ignored\n",
1548  hostname);
1549  GNUNET_free (req);
1550  return;
1551  }
1552 }
static char * hostname
Our hostname; we give this to all the peers we start.
uint32_t GNUNET_CRYPTO_random_u32(enum GNUNET_CRYPTO_Quality mode, uint32_t i)
Produce a random value.
@ GNUNET_CRYPTO_QUALITY_NONCE
Randomness for IVs etc.
int GNUNET_DNSPARSER_check_name(const char *name)
Check if a hostname in UTF-8 format can be coded into valid IDNA.
Definition: dnsparser.c:79
enum GNUNET_GenericReturnValue GNUNET_CONTAINER_multihashmap_put(struct GNUNET_CONTAINER_MultiHashMap *map, const struct GNUNET_HashCode *key, void *value, enum GNUNET_CONTAINER_MultiHashMapOption opt)
Store a key-value pair in the map.
@ GNUNET_CONTAINER_MULTIHASHMAPOPTION_UNIQUE_ONLY
There must only be one value per key; storing a value should fail if a value under the same key alrea...

References GNUNET_CONTAINER_multihashmap_put(), GNUNET_CONTAINER_MULTIHASHMAPOPTION_UNIQUE_ONLY, GNUNET_CRYPTO_hash(), GNUNET_CRYPTO_QUALITY_NONCE, GNUNET_CRYPTO_random_u32(), GNUNET_DNSPARSER_check_name(), GNUNET_ERROR_TYPE_ERROR, GNUNET_ERROR_TYPE_WARNING, GNUNET_free, GNUNET_log, GNUNET_malloc, GNUNET_memcpy, GNUNET_OK, Request::hostname, hostname, Request::id, ns_pending, rejects, zone, Request::zone, and zone_head.

Referenced by process_stdin().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ move_to_queue()

static int move_to_queue ( void *  cls,
const struct GNUNET_HashCode key,
void *  value 
)
static

We have completed the initial iteration over the namestore's database.

This function is called on each of the remaining records in move_to_queue to queue() them, as we will simply not find existing records for them any longer.

Parameters
clsNULL
keyunused
valuea struct Request
Returns
GNUNET_OK (continue to iterate)

Definition at line 1567 of file gnunet-zoneimport.c.

1568 {
1569  struct Request *req = value;
1570 
1571  (void) cls;
1572  (void) key;
1573  insert_sorted (req);
1574  return GNUNET_OK;
1575 }

References GNUNET_OK, insert_sorted(), key, and value.

Referenced by iterate_zones().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ process_stdin()

static void process_stdin ( void *  cls)
static

Begin processing hostnames from stdin.

Parameters
clsNULL

Definition at line 1653 of file gnunet-zoneimport.c.

1654 {
1655  static struct GNUNET_TIME_Absolute last;
1656  static uint64_t idot;
1657  char hn[256];
1658 
1659  (void) cls;
1660  t = NULL;
1661  if (NULL != id)
1662  {
1664  id = NULL;
1665  }
1666  while (NULL != fgets (hn, sizeof(hn), stdin))
1667  {
1668  if (strlen (hn) > 0)
1669  hn[strlen (hn) - 1] = '\0'; /* eat newline */
1670  if (0 == idot)
1671  last = GNUNET_TIME_absolute_get ();
1672  idot++;
1673  if (0 == idot % 100000)
1674  {
1675  struct GNUNET_TIME_Relative delta;
1676 
1678  last = GNUNET_TIME_absolute_get ();
1679  fprintf (stderr,
1680  "Read 100000 domain names in %s\n",
1682  GNUNET_STATISTICS_set (stats, "# domain names provided", idot, GNUNET_NO);
1683  }
1684  queue (hn);
1685  }
1686  fprintf (stderr,
1687  "Done reading %llu domain names\n",
1688  (unsigned long long) idot);
1689  GNUNET_STATISTICS_set (stats, "# domain names provided", idot, GNUNET_NO);
1690  iterate_zones (NULL);
1691 }
static void queue(const char *hostname)
Add hostname to the list of requests to be made.

References delta, GNUNET_IDENTITY_disconnect(), GNUNET_NO, GNUNET_STATISTICS_set(), GNUNET_STRINGS_relative_time_to_string(), GNUNET_TIME_absolute_get(), GNUNET_TIME_absolute_get_duration(), GNUNET_YES, iterate_zones(), queue(), stats, and t.

Referenced by identity_cb().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ identity_cb()

static void identity_cb ( void *  cls,
struct GNUNET_IDENTITY_Ego ego,
void **  ctx,
const char *  name 
)
static

Method called to inform about the egos of this peer.

When used with GNUNET_IDENTITY_connect, this function is initially called for all egos and then again whenever a ego's name changes or if it is deleted. At the end of the initial pass over all egos, the function is once called with 'NULL' for ego. That does NOT mean that the callback won't be invoked in the future or that there was an error.

When used with GNUNET_IDENTITY_create or GNUNET_IDENTITY_get, this function is only called ONCE, and 'NULL' being passed in ego does indicate an error (for example because name is taken or no default value is known). If ego is non-NULL and if '*ctx' is set in those callbacks, the value WILL be passed to a subsequent call to the identity callback of GNUNET_IDENTITY_connect (if that one was not NULL).

When an identity is renamed, this function is called with the (known) ego but the NEW name.

When an identity is deleted, this function is called with the (known) ego and "NULL" for the name. In this case, the ego is henceforth invalid (and the ctx should also be cleaned up).

Parameters
clsclosure
egoego handle, NULL for end of list
ctxcontext for application to store data for this ego (during the lifetime of this process, initially NULL)
namename assigned by the user for this ego, NULL if the user just deleted the ego and it must thus no longer be used

Definition at line 1728 of file gnunet-zoneimport.c.

1732 {
1733  (void) cls;
1734  (void) ctx;
1735 
1736  if (NULL == ego)
1737  {
1738  /* end of iteration */
1739  if (NULL == zone_head)
1740  {
1741  GNUNET_log (GNUNET_ERROR_TYPE_ERROR, "No zone found\n");
1743  return;
1744  }
1745  /* zone_head non-null, process hostnames from stdin */
1747  return;
1748  }
1749  if (NULL != name)
1750  {
1751  struct Zone *zone;
1752 
1753  zone = GNUNET_new (struct Zone);
1755  zone->domain = GNUNET_strdup (name);
1757  }
1758 }
static void process_stdin(void *cls)
Begin processing hostnames from stdin.
const struct GNUNET_IDENTITY_PrivateKey * GNUNET_IDENTITY_ego_get_private_key(const struct GNUNET_IDENTITY_Ego *ego)
Obtain the ECC key associated with a ego.
Definition: identity_api.c:560
#define GNUNET_strdup(a)
Wrapper around GNUNET_xstrdup_.
#define GNUNET_new(type)
Allocate a struct or union of the given type.
void GNUNET_SCHEDULER_shutdown(void)
Request the shutdown of a scheduler.
Definition: scheduler.c:562
const char * name

References ctx, GNUNET_CONTAINER_DLL_insert, GNUNET_ERROR_TYPE_ERROR, GNUNET_IDENTITY_ego_get_private_key(), GNUNET_log, GNUNET_new, GNUNET_SCHEDULER_add_now(), GNUNET_SCHEDULER_shutdown(), GNUNET_strdup, name, process_stdin(), t, zone, zone_head, and zone_tail.

Referenced by run().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ run()

static void run ( void *  cls,
char *const *  args,
const char *  cfgfile,
const struct GNUNET_CONFIGURATION_Handle cfg 
)
static

Process requests from the queue, then if the queue is not empty, try again.

Parameters
clsNULL
argsremaining command-line arguments
cfgfilename of the configuration file used (for saving, can be NULL!)
cfgconfiguration

Definition at line 1771 of file gnunet-zoneimport.c.

1775 {
1776  (void) cls;
1777  (void) args;
1778  (void) cfgfile;
1779  stats = GNUNET_STATISTICS_create ("zoneimport", cfg);
1782  if (NULL == ns_pending)
1783  {
1784  fprintf (stderr, "Failed to allocate memory for main hash map\n");
1785  return;
1786  }
1787  ctx = GNUNET_DNSSTUB_start (256);
1788  if (NULL == ctx)
1789  {
1790  fprintf (stderr, "Failed to initialize GNUnet DNS STUB\n");
1791  return;
1792  }
1793  if (NULL == args[0])
1794  {
1795  fprintf (stderr,
1796  "You must provide a list of DNS resolvers on the command line\n");
1797  return;
1798  }
1799  for (unsigned int i = 0; NULL != args[i]; i++)
1800  {
1802  {
1803  fprintf (stderr, "Failed to use `%s' for DNS resolver\n", args[i]);
1804  return;
1805  }
1806  }
1807 
1808 
1811  if (NULL == ns)
1812  {
1814  return;
1815  }
1816  id = GNUNET_IDENTITY_connect (cfg, &identity_cb, NULL);
1817 }
static const struct GNUNET_CONFIGURATION_Handle * cfg
Configuration we are using.
Definition: gnunet-abd.c:36
static unsigned int map_size
Command-line argument specifying desired size of the hash map with all of our pending names.
static void do_shutdown(void *cls)
Clean up and terminate the process.
static void identity_cb(void *cls, struct GNUNET_IDENTITY_Ego *ego, void **ctx, const char *name)
Method called to inform about the egos of this peer.
int GNUNET_DNSSTUB_add_dns_ip(struct GNUNET_DNSSTUB_Context *ctx, const char *dns_ip)
Add nameserver for use by the DNSSTUB.
Definition: dnsstub.c:613
struct GNUNET_DNSSTUB_Context * GNUNET_DNSSTUB_start(unsigned int num_sockets)
Start a DNS stub resolver.
Definition: dnsstub.c:586
struct GNUNET_CONTAINER_MultiHashMap * GNUNET_CONTAINER_multihashmap_create(unsigned int len, int do_not_copy_keys)
Create a multi hash map.
struct GNUNET_CONTAINER_Heap * GNUNET_CONTAINER_heap_create(enum GNUNET_CONTAINER_HeapOrder order)
Create a new heap.
@ GNUNET_CONTAINER_HEAP_ORDER_MIN
Heap with the minimum cost at the root.
struct GNUNET_IDENTITY_Handle * GNUNET_IDENTITY_connect(const struct GNUNET_CONFIGURATION_Handle *cfg, GNUNET_IDENTITY_Callback cb, void *cb_cls)
Connect to the identity service.
Definition: identity_api.c:531
struct GNUNET_NAMESTORE_Handle * GNUNET_NAMESTORE_connect(const struct GNUNET_CONFIGURATION_Handle *cfg)
Connect to the namestore service.
struct GNUNET_SCHEDULER_Task * GNUNET_SCHEDULER_add_shutdown(GNUNET_SCHEDULER_TaskCallback task, void *task_cls)
Schedule a new task to be run on shutdown, that is when a CTRL-C signal is received,...
Definition: scheduler.c:1334
struct GNUNET_STATISTICS_Handle * GNUNET_STATISTICS_create(const char *subsystem, const struct GNUNET_CONFIGURATION_Handle *cfg)
Get handle for the statistics service.

References consensus-simulation::args, cfg, ctx, do_shutdown(), GNUNET_CONTAINER_heap_create(), GNUNET_CONTAINER_HEAP_ORDER_MIN, GNUNET_CONTAINER_multihashmap_create(), GNUNET_DNSSTUB_add_dns_ip(), GNUNET_DNSSTUB_start(), GNUNET_IDENTITY_connect(), GNUNET_NAMESTORE_connect(), GNUNET_NO, GNUNET_OK, GNUNET_SCHEDULER_add_shutdown(), GNUNET_SCHEDULER_shutdown(), GNUNET_STATISTICS_create(), identity_cb(), map_size, ns, ns_pending, req_heap, and stats.

Referenced by main().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ main()

int main ( int  argc,
char *const *  argv 
)

Call with IP address of resolver to query.

Parameters
argcshould be 2
argv[1]should contain IP address
Returns
0 on success

Definition at line 1828 of file gnunet-zoneimport.c.

1829 {
1832  "size",
1833  "MAPSIZE",
1834  gettext_noop (
1835  "size to use for the main hash map"),
1836  &map_size),
1838  'm',
1839  "minimum-expiration",
1840  "RELATIVETIME",
1841  gettext_noop ("minimum expiration time we assume for imported records"),
1844  int ret;
1845 
1846  if (GNUNET_OK != GNUNET_STRINGS_get_utf8_args (argc, argv, &argc, &argv))
1847  return 2;
1848  if (GNUNET_OK != (ret = GNUNET_PROGRAM_run (argc,
1849  argv,
1850  "gnunet-zoneimport",
1851  "import DNS zone into namestore",
1852  options,
1853  &run,
1854  NULL)))
1855  return ret;
1856  GNUNET_free_nz ((void *) argv);
1857  fprintf (stderr,
1858  "Rejected %u names, had %u cached, did %u lookups, stored %u record sets\n"
1859  "Found %u records, %u lookups failed, %u/%u pending on shutdown\n",
1860  rejects,
1861  cached,
1862  lookups,
1863  record_sets,
1864  records,
1865  failures,
1866  pending,
1867  pending_rs);
1868  return 0;
1869 }
struct GNUNET_GETOPT_CommandLineOption GNUNET_GETOPT_OPTION_END
Definition: 002.c:13
struct GNUNET_GETOPT_CommandLineOption options[]
Definition: 002.c:5
#define gettext_noop(String)
Definition: gettext.h:70
static void run(void *cls, char *const *args, const char *cfgfile, const struct GNUNET_CONFIGURATION_Handle *cfg)
Process requests from the queue, then if the queue is not empty, try again.
struct GNUNET_GETOPT_CommandLineOption GNUNET_GETOPT_option_uint(char shortName, const char *name, const char *argumentHelp, const char *description, unsigned int *val)
Allow user to specify an unsigned int.
struct GNUNET_GETOPT_CommandLineOption GNUNET_GETOPT_option_relative_time(char shortName, const char *name, const char *argumentHelp, const char *description, struct GNUNET_TIME_Relative *val)
Allow user to specify a struct GNUNET_TIME_Relative (using human-readable "fancy" time).
#define GNUNET_free_nz(ptr)
Wrapper around free.
enum GNUNET_GenericReturnValue GNUNET_PROGRAM_run(int argc, char *const *argv, const char *binaryName, const char *binaryHelp, const struct GNUNET_GETOPT_CommandLineOption *options, GNUNET_PROGRAM_Main task, void *task_cls)
Run a standard GNUnet command startup sequence (initialize loggers and configuration,...
Definition: program.c:400
enum GNUNET_GenericReturnValue GNUNET_STRINGS_get_utf8_args(int argc, char *const *argv, int *u8argc, char *const **u8argv)
Returns utf-8 encoded arguments.
Definition: strings.c:1222
Definition of a command line option.

References cached, failures, gettext_noop, GNUNET_free_nz, GNUNET_GETOPT_OPTION_END, GNUNET_GETOPT_option_relative_time(), GNUNET_GETOPT_option_uint(), GNUNET_OK, GNUNET_PROGRAM_run(), GNUNET_STRINGS_get_utf8_args(), lookups, map_size, minimum_expiration_time, options, pending, pending_rs, record_sets, records, rejects, ret, and run().

Here is the call graph for this function:

Variable Documentation

◆ minimum_expiration_time

struct GNUNET_TIME_Relative minimum_expiration_time
static

How long do DNS records have to last at least after being imported?

Definition at line 1 of file gnunet-zoneimport.c.

Referenced by check_for_glue(), main(), and process_record().

◆ map_size

unsigned int map_size = 1024
static

Command-line argument specifying desired size of the hash map with all of our pending names.

Usually, we use an automatically growing map, but this is only OK up to about a million entries. Above that number, the user must explicitly specify the size at startup.

Definition at line 213 of file gnunet-zoneimport.c.

Referenced by main(), and run().

◆ id

struct GNUNET_IDENTITY_Handle* id
static

Handle to the identity service.

Definition at line 218 of file gnunet-zoneimport.c.

◆ ns

struct GNUNET_NAMESTORE_Handle* ns
static

Namestore handle.

Definition at line 223 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), iterate_zones(), process_result(), and run().

◆ stats

struct GNUNET_STATISTICS_Handle* stats
static

Handle to the statistics service.

Definition at line 228 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), iterate_zones(), ns_lookup_result_cb(), process_record(), process_result(), process_stdin(), run(), and store_completed_cb().

◆ ctx

struct GNUNET_DNSSTUB_Context* ctx
static

Context for DNS resolution.

Definition at line 233 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), identity_cb(), process_queue(), and run().

◆ pending

unsigned int pending
static

The number of DNS queries that are outstanding.

Definition at line 238 of file gnunet-zoneimport.c.

Referenced by main(), process_queue(), and process_result().

◆ pending_rs

unsigned int pending_rs
static

The number of NAMESTORE record store operations that are outstanding.

Definition at line 243 of file gnunet-zoneimport.c.

Referenced by main(), process_queue(), process_result(), and store_completed_cb().

◆ lookups

unsigned int lookups
static

Number of lookups we performed overall.

Definition at line 248 of file gnunet-zoneimport.c.

Referenced by main(), and process_queue().

◆ cached

unsigned int cached
static

Number of records we had cached.

Definition at line 253 of file gnunet-zoneimport.c.

Referenced by main(), and ns_lookup_result_cb().

◆ rejects

unsigned int rejects
static

How many hostnames did we reject (malformed).

Definition at line 258 of file gnunet-zoneimport.c.

Referenced by build_dns_query(), main(), and queue().

◆ failures

unsigned int failures
static

Number of lookups that failed.

Definition at line 263 of file gnunet-zoneimport.c.

Referenced by main(), and process_result().

◆ records

unsigned int records
static

Number of records we found.

Definition at line 268 of file gnunet-zoneimport.c.

Referenced by main(), and process_record().

◆ record_sets

unsigned int record_sets
static

Number of record sets given to namestore.

Definition at line 273 of file gnunet-zoneimport.c.

Referenced by main(), and process_result().

◆ req_heap

struct GNUNET_CONTAINER_Heap* req_heap
static

Heap of all requests to perform, sorted by the time we should next do the request (i.e.

by expires).

Definition at line 279 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), insert_sorted(), process_queue(), and run().

◆ req_head

struct Request* req_head
static

Active requests are kept in a DLL.

Definition at line 284 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), process_queue(), and process_result().

◆ req_tail

struct Request* req_tail
static

Active requests are kept in a DLL.

Definition at line 289 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), process_queue(), and process_result().

◆ t

◆ ns_pending

struct GNUNET_CONTAINER_MultiHashMap* ns_pending
static

Hash map of requests for which we may still get a response from the namestore.

Set to NULL once the initial namestore iteration is done.

Definition at line 301 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), iterate_zones(), ns_lookup_result_cb(), queue(), and run().

◆ zone_it

struct GNUNET_NAMESTORE_ZoneIterator* zone_it
static

Current zone iteration handle.

Definition at line 306 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), iterate_zones(), ns_lookup_error_cb(), and ns_lookup_result_cb().

◆ zone_head

struct Zone* zone_head
static

Head of list of zones we are managing.

Definition at line 311 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), identity_cb(), iterate_zones(), and queue().

◆ zone_tail

struct Zone* zone_tail
static

Tail of list of zones we are managing.

Definition at line 316 of file gnunet-zoneimport.c.

Referenced by do_shutdown(), identity_cb(), and iterate_zones().

◆ ns_iterator_trigger_next

uint64_t ns_iterator_trigger_next
static

After how many more results must ns_lookup_result_cb() ask the namestore for more?

Definition at line 322 of file gnunet-zoneimport.c.

Referenced by iterate_zones(), ns_lookup_error_cb(), and ns_lookup_result_cb().

◆ total_dns_latency_cnt

uint64_t total_dns_latency_cnt
static

Number of DNS requests counted in latency total.

Definition at line 327 of file gnunet-zoneimport.c.

Referenced by process_result().

◆ total_dns_latency

struct GNUNET_TIME_Relative total_dns_latency
static

Sum of DNS latencies observed.

Definition at line 327 of file gnunet-zoneimport.c.

Referenced by process_result().

◆ total_reg_proc_dns

uint64_t total_reg_proc_dns
static

Number of records processed (DNS lookup, no NAMESTORE) in total.

Definition at line 337 of file gnunet-zoneimport.c.

Referenced by iterate_zones(), process_result(), and store_completed_cb().

◆ total_reg_proc_dns_ns

uint64_t total_reg_proc_dns_ns
static

Number of records processed (DNS lookup, with NAMESTORE) in total.

Definition at line 342 of file gnunet-zoneimport.c.

Referenced by iterate_zones(), and store_completed_cb().

◆ start_time_reg_proc

struct GNUNET_TIME_Absolute start_time_reg_proc
static

Start time of the regular processing.

Definition at line 342 of file gnunet-zoneimport.c.

Referenced by iterate_zones(), and store_completed_cb().

◆ sleep_time_reg_proc

struct GNUNET_TIME_Absolute sleep_time_reg_proc
static

Last time we worked before going idle.

Definition at line 342 of file gnunet-zoneimport.c.

Referenced by insert_sorted(), process_queue(), process_result(), and store_completed_cb().

◆ idle_time

struct GNUNET_TIME_Relative idle_time
static

Time we slept just waiting for work.

Definition at line 342 of file gnunet-zoneimport.c.

Referenced by process_queue(), and store_completed_cb().