GNUnet  0.16.x
Data Structures | Macros | Functions | Variables
gnunet-service-fs_pr.c File Reference

API to handle pending requests. More...

#include "platform.h"
#include "gnunet_util_lib.h"
#include "gnunet_load_lib.h"
#include "gnunet-service-fs.h"
#include "gnunet-service-fs_cp.h"
#include "gnunet-service-fs_indexing.h"
#include "gnunet-service-fs_pe.h"
#include "gnunet-service-fs_pr.h"
#include "gnunet-service-fs_cadet.h"
Include dependency graph for gnunet-service-fs_pr.c:

Go to the source code of this file.

Data Structures

struct  GSF_PendingRequest
 An active request. More...
 
struct  ProcessReplyClosure
 Closure for process_reply() function. More...
 
struct  PutMigrationContext
 Context for put_migration_continuation(). More...
 

Macros

#define DHT_GET_REPLICATION   5
 Desired replication level for GETs. More...
 
#define MAX_DATASTORE_QUEUE   (16 * MAX_QUEUE_PER_PEER)
 Maximum size of the datastore queue for P2P operations. More...
 
#define CONTENT_BANDWIDTH_VALUE   800
 Bandwidth value of a 0-priority content (must be fairly high compared to query since content is typically significantly larger – and more valuable since it can take many queries to get one piece of content). More...
 
#define MAX_RESULTS   (100 * 1024)
 Hard limit on the number of results we may get from the datastore per query. More...
 
#define INSANE_STATISTICS   GNUNET_NO
 Collect an instance number of statistics? May cause excessive IPC. More...
 
#define CADET_RETRY_MAX   3
 If obtaining a block via cadet fails, how often do we retry it before giving up for good (and sticking to non-anonymous transfer)? More...
 

Functions

static void refresh_bloomfilter (enum GNUNET_BLOCK_Type type, struct GSF_PendingRequest *pr)
 Recalculate our bloom filter for filtering replies. More...
 
struct GSF_PendingRequestGSF_pending_request_create_ (enum GSF_PendingRequestOptions options, enum GNUNET_BLOCK_Type type, const struct GNUNET_HashCode *query, const struct GNUNET_PeerIdentity *target, const char *bf_data, size_t bf_size, uint32_t mingle, uint32_t anonymity_level, uint32_t priority, int32_t ttl, GNUNET_PEER_Id sender_pid, GNUNET_PEER_Id origin_pid, const struct GNUNET_HashCode *replies_seen, unsigned int replies_seen_count, GSF_PendingRequestReplyHandler rh, void *rh_cls)
 Create a new pending request. More...
 
struct GSF_PendingRequestDataGSF_pending_request_get_data_ (struct GSF_PendingRequest *pr)
 Obtain the public data associated with a pending request. More...
 
int GSF_pending_request_is_compatible_ (struct GSF_PendingRequest *pra, struct GSF_PendingRequest *prb)
 Test if two pending requests are compatible (would generate the same query modulo filters and should thus be processed jointly). More...
 
void GSF_pending_request_update_ (struct GSF_PendingRequest *pr, const struct GNUNET_HashCode *replies_seen, unsigned int replies_seen_count)
 Update a given pending request with additional replies that have been seen. More...
 
struct GNUNET_MQ_EnvelopeGSF_pending_request_get_message_ (struct GSF_PendingRequest *pr)
 Generate the message corresponding to the given pending request for transmission to other peers. More...
 
static int clean_request (void *cls, const struct GNUNET_HashCode *key, void *value)
 Iterator to free pending requests. More...
 
void GSF_pending_request_cancel_ (struct GSF_PendingRequest *pr, int full_cleanup)
 Explicitly cancel a pending request. More...
 
void GSF_iterate_pending_requests_ (GSF_PendingRequestIterator it, void *cls)
 Iterate over all pending requests. More...
 
static void update_request_performance_data (struct ProcessReplyClosure *prq, struct GSF_PendingRequest *pr)
 Update the performance data for the sender (if any) since the sender successfully answered one of our queries. More...
 
static enum GNUNET_GenericReturnValue process_reply (void *cls, const struct GNUNET_HashCode *key, void *value)
 We have received a reply; handle it! More...
 
static void put_migration_continuation (void *cls, int success, struct GNUNET_TIME_Absolute min_expiration, const char *msg)
 Continuation called to notify client about result of the operation. More...
 
static int test_put_load_too_high (uint32_t priority)
 Test if the DATABASE (PUT) load on this peer is too high to even consider processing the query at all. More...
 
static void handle_dht_reply (void *cls, struct GNUNET_TIME_Absolute exp, const struct GNUNET_HashCode *key, const struct GNUNET_DHT_PathElement *get_path, unsigned int get_path_length, const struct GNUNET_DHT_PathElement *put_path, unsigned int put_path_length, enum GNUNET_BLOCK_Type type, size_t size, const void *data)
 Iterator called on each result obtained for a DHT operation that expects a reply. More...
 
void GSF_dht_lookup_ (struct GSF_PendingRequest *pr)
 Consider looking up the data in the DHT (anonymity-level permitting). More...
 
static void cadet_reply_proc (void *cls, enum GNUNET_BLOCK_Type type, struct GNUNET_TIME_Absolute expiration, size_t data_size, const void *data)
 Function called with a reply from the cadet. More...
 
void GSF_cadet_lookup_ (struct GSF_PendingRequest *pr)
 Consider downloading via cadet (if possible) More...
 
static void warn_delay_task (void *cls)
 Task that issues a warning if the datastore lookup takes too long. More...
 
static void odc_warn_delay_task (void *cls)
 Task that issues a warning if the datastore lookup takes too long. More...
 
static void call_continuation (struct GSF_PendingRequest *pr)
 
static void no_more_local_results (struct GSF_PendingRequest *pr)
 
static void process_local_reply (void *cls, const struct GNUNET_HashCode *key, size_t size, const void *data, enum GNUNET_BLOCK_Type type, uint32_t priority, uint32_t anonymity, uint32_t replication, struct GNUNET_TIME_Absolute expiration, uint64_t uid)
 We're processing (local) results for a search request from another peer. More...
 
static void start_local_query (struct GSF_PendingRequest *pr, uint64_t next_uid, bool random)
 
int GSF_pending_request_test_target_ (struct GSF_PendingRequest *pr, const struct GNUNET_PeerIdentity *target)
 Is the given target a legitimate peer for forwarding the given request? More...
 
void GSF_local_lookup_ (struct GSF_PendingRequest *pr, GSF_LocalLookupContinuation cont, void *cont_cls)
 Look up the request in the local datastore. More...
 
void handle_p2p_put (void *cls, const struct PutMessage *put)
 Handle P2P "CONTENT" message. More...
 
int GSF_pending_request_test_active_ (struct GSF_PendingRequest *pr)
 Check if the given request is still active. More...
 
void GSF_pending_request_init_ ()
 Setup the subsystem. More...
 
void GSF_pending_request_done_ ()
 Shutdown the subsystem. More...
 

Variables

static struct GNUNET_CONTAINER_MultiHashMappr_map
 All pending requests, ordered by the query. More...
 
static struct GNUNET_LOAD_Valuedatastore_put_load
 Datastore 'PUT' load tracking. More...
 
static int active_to_migration
 Are we allowed to migrate content to this peer. More...
 
static struct GNUNET_CONTAINER_Heaprequests_by_expiration_heap
 Heap with the request that will expire next at the top. More...
 
static unsigned long long max_pending_requests = (32 * 1024)
 Maximum number of requests (from other peers, overall) that we're willing to have pending at any given point in time. More...
 

Detailed Description

API to handle pending requests.

Author
Christian Grothoff

Definition in file gnunet-service-fs_pr.c.

Macro Definition Documentation

◆ DHT_GET_REPLICATION

#define DHT_GET_REPLICATION   5

Desired replication level for GETs.

Definition at line 40 of file gnunet-service-fs_pr.c.

◆ MAX_DATASTORE_QUEUE

#define MAX_DATASTORE_QUEUE   (16 * MAX_QUEUE_PER_PEER)

Maximum size of the datastore queue for P2P operations.

Needs to be large enough to queue MAX_QUEUE_PER_PEER operations for roughly the number of active (connected) peers.

Definition at line 47 of file gnunet-service-fs_pr.c.

◆ CONTENT_BANDWIDTH_VALUE

#define CONTENT_BANDWIDTH_VALUE   800

Bandwidth value of a 0-priority content (must be fairly high compared to query since content is typically significantly larger – and more valuable since it can take many queries to get one piece of content).

Definition at line 55 of file gnunet-service-fs_pr.c.

◆ MAX_RESULTS

#define MAX_RESULTS   (100 * 1024)

Hard limit on the number of results we may get from the datastore per query.

Definition at line 60 of file gnunet-service-fs_pr.c.

◆ INSANE_STATISTICS

#define INSANE_STATISTICS   GNUNET_NO

Collect an instance number of statistics? May cause excessive IPC.

Definition at line 65 of file gnunet-service-fs_pr.c.

◆ CADET_RETRY_MAX

#define CADET_RETRY_MAX   3

If obtaining a block via cadet fails, how often do we retry it before giving up for good (and sticking to non-anonymous transfer)?

Definition at line 71 of file gnunet-service-fs_pr.c.

Function Documentation

◆ refresh_bloomfilter()

static void refresh_bloomfilter ( enum GNUNET_BLOCK_Type  type,
struct GSF_PendingRequest pr 
)
static

Recalculate our bloom filter for filtering replies.

This function will create a new bloom filter from scratch, so it should only be called if we have no bloomfilter at all (and hence can create a fresh one of minimal size without problems) OR if our peer is the initiator (in which case we may resize to larger than minimum size).

Parameters
typetype of the request
prrequest for which the BF is to be recomputed

Definition at line 250 of file gnunet-service-fs_pr.c.

252 {
253  if (NULL != pr->bg)
254  {
256  pr->bg = NULL;
257  }
259  return; /* no need */
260  pr->bg =
262  type,
265  UINT32_MAX),
266  NULL,
267  0,
268  "seen-set-size",
269  pr->replies_seen_count,
270  NULL);
271  if (NULL == pr->bg)
272  return;
275  pr->replies_seen,
276  pr->replies_seen_count));
277 }
struct GNUNET_BLOCK_Context * GSF_block_ctx
Our block context.
@ GNUNET_OK
Definition: gnunet_common.h:95
@ GNUNET_BLOCK_TYPE_FS_UBLOCK
Type of a block representing any type of search result (universal).
enum GNUNET_GenericReturnValue GNUNET_BLOCK_group_set_seen(struct GNUNET_BLOCK_Group *bg, const struct GNUNET_HashCode *seen_results, unsigned int seen_results_count)
Update block group to filter out the given results.
Definition: block.c:343
void GNUNET_BLOCK_group_destroy(struct GNUNET_BLOCK_Group *bg)
Destroy resources used by a block group.
Definition: block.c:173
struct GNUNET_BLOCK_Group * GNUNET_BLOCK_group_create(struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, uint32_t nonce, const void *raw_data, size_t raw_data_size,...)
Create a new block group.
Definition: block.c:228
uint32_t GNUNET_CRYPTO_random_u32(enum GNUNET_CRYPTO_Quality mode, uint32_t i)
Produce a random value.
@ GNUNET_CRYPTO_QUALITY_WEAK
No good quality of the operation is needed (i.e., random numbers can be pseudo-random).
#define GNUNET_break(cond)
Use this for internal assertion violations that are not fatal (can be handled) but should not occur.
struct GNUNET_BLOCK_Group * bg
Block group for filtering replies we've already seen.
unsigned int replies_seen_count
Number of valid entries in the 'replies_seen' array.
struct GNUNET_HashCode * replies_seen
Array of hash codes of replies we've already seen.
enum GNUNET_TESTBED_UnderlayLinkModelType type
the type of this model

References GSF_PendingRequest::bg, GNUNET_BLOCK_group_create(), GNUNET_BLOCK_group_destroy(), GNUNET_BLOCK_group_set_seen(), GNUNET_BLOCK_TYPE_FS_UBLOCK, GNUNET_break, GNUNET_CRYPTO_QUALITY_WEAK, GNUNET_CRYPTO_random_u32(), GNUNET_OK, GSF_block_ctx, GSF_PendingRequest::replies_seen, GSF_PendingRequest::replies_seen_count, and type.

Referenced by GSF_pending_request_create_(), and GSF_pending_request_update_().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_pending_request_create_()

struct GSF_PendingRequest* GSF_pending_request_create_ ( enum GSF_PendingRequestOptions  options,
enum GNUNET_BLOCK_Type  type,
const struct GNUNET_HashCode query,
const struct GNUNET_PeerIdentity target,
const char *  bf_data,
size_t  bf_size,
uint32_t  mingle,
uint32_t  anonymity_level,
uint32_t  priority,
int32_t  ttl,
GNUNET_PEER_Id  sender_pid,
GNUNET_PEER_Id  origin_pid,
const struct GNUNET_HashCode replies_seen,
unsigned int  replies_seen_count,
GSF_PendingRequestReplyHandler  rh,
void *  rh_cls 
)

Create a new pending request.

Parameters
optionsrequest options
typetype of the block that is being requested
querykey for the lookup
targetpreferred target for the request, NULL for none
bf_dataraw data for bloom filter for known replies, can be NULL
bf_sizenumber of bytes in bf_data
minglemingle value for bf
anonymity_leveldesired anonymity level
prioritymaximum outgoing cumulative request priority to use
ttlcurrent time-to-live for the request
sender_pidpeer ID to use for the sender when forwarding, 0 for none
origin_pidpeer ID of origin of query (do not loop back)
replies_seenhash codes of known local replies
replies_seen_countsize of the replies_seen array
rhhandle to call when we get a reply
rh_clsclosure for rh
Returns
handle for the new pending request

Definition at line 302 of file gnunet-service-fs_pr.c.

318 {
319  struct GSF_PendingRequest *pr;
320  struct GSF_PendingRequest *dpr;
321  size_t extra;
322  struct GNUNET_HashCode *eptr;
323 
325  "Creating request handle for `%s' of type %d\n",
326  GNUNET_h2s (query),
327  type);
328 #if INSANE_STATISTICS
330  gettext_noop ("# Pending requests created"),
331  1,
332  GNUNET_NO);
333 #endif
334  extra = 0;
335  if (NULL != target)
336  extra += sizeof(struct GNUNET_PeerIdentity);
337  pr = GNUNET_malloc (sizeof(struct GSF_PendingRequest) + extra);
338  pr->public_data.query = *query;
339  eptr = (struct GNUNET_HashCode *) &pr[1];
340  if (NULL != target)
341  {
342  pr->public_data.target = (struct GNUNET_PeerIdentity *) eptr;
343  GNUNET_memcpy (eptr, target, sizeof(struct GNUNET_PeerIdentity));
344  }
346  pr->public_data.priority = priority;
347  pr->public_data.original_priority = priority;
349  pr->public_data.type = type;
351  pr->sender_pid = sender_pid;
352  pr->origin_pid = origin_pid;
353  pr->rh = rh;
354  pr->rh_cls = rh_cls;
355  GNUNET_assert ((sender_pid != 0) || (0 == (options & GSF_PRO_FORWARD_ONLY)));
356  if (ttl >= 0)
359  else
363  (uint32_t) (-ttl)));
364  if (replies_seen_count > 0)
365  {
366  pr->replies_seen_size = replies_seen_count;
367  pr->replies_seen =
370  replies_seen,
371  replies_seen_count * sizeof(struct GNUNET_HashCode));
372  pr->replies_seen_count = replies_seen_count;
373  }
374  if ((NULL != bf_data) &&
376  {
378  pr->public_data.type,
379  mingle,
380  bf_data,
381  bf_size,
382  "seen-set-size",
383  0,
384  NULL);
385  }
386  else if ((replies_seen_count > 0) &&
388  {
390  }
392  &pr->public_data.query,
393  pr,
396  {
398  pr,
400  /* make sure we don't track too many requests */
403  {
405  GNUNET_assert (NULL != dpr);
406  if (pr == dpr)
407  break; /* let the request live briefly... */
408  if (NULL != dpr->rh)
409  dpr->rh (dpr->rh_cls,
411  dpr,
412  UINT32_MAX,
416  NULL,
417  0);
419  }
420  }
422  gettext_noop ("# Pending requests active"),
423  1,
424  GNUNET_NO);
425  return pr;
426 }
struct GNUNET_GETOPT_CommandLineOption options[]
Definition: 002.c:5
#define gettext_noop(String)
Definition: gettext.h:69
static unsigned int anonymity_level
Anonymity level option to use for publishing.
struct GNUNET_STATISTICS_Handle * GSF_stats
Handle for reporting statistics.
static unsigned long long max_pending_requests
Maximum number of requests (from other peers, overall) that we're willing to have pending at any give...
static void refresh_bloomfilter(enum GNUNET_BLOCK_Type type, struct GSF_PendingRequest *pr)
Recalculate our bloom filter for filtering replies.
static struct GNUNET_CONTAINER_Heap * requests_by_expiration_heap
Heap with the request that will expire next at the top.
static struct GNUNET_CONTAINER_MultiHashMap * pr_map
All pending requests, ordered by the query.
void GSF_pending_request_cancel_(struct GSF_PendingRequest *pr, int full_cleanup)
Explicitly cancel a pending request.
@ GSF_PRO_FORWARD_ONLY
Request must only be forwarded (no routing)
@ GSF_PRO_BLOOMFILTER_FULL_REFRESH
Request is allowed to refresh bloomfilter and change mingle value.
@ GSF_PRO_REQUEST_NEVER_EXPIRES
Request persists indefinitely (no expiration).
#define GNUNET_log(kind,...)
#define GNUNET_memcpy(dst, src, n)
Call memcpy() but check for n being 0 first.
@ GNUNET_YES
Definition: gnunet_common.h:97
@ GNUNET_NO
Definition: gnunet_common.h:94
@ GNUNET_BLOCK_TYPE_ANY
Identifier for any block.
@ GNUNET_BLOCK_REPLY_TYPE_NOT_SUPPORTED
Specified block type not supported by any plugin.
int GNUNET_CONTAINER_multihashmap_put(struct GNUNET_CONTAINER_MultiHashMap *map, const struct GNUNET_HashCode *key, void *value, enum GNUNET_CONTAINER_MultiHashMapOption opt)
Store a key-value pair in the map.
@ GNUNET_CONTAINER_MULTIHASHMAPOPTION_MULTIPLE
Allow multiple values with the same key.
struct GNUNET_CONTAINER_HeapNode * GNUNET_CONTAINER_heap_insert(struct GNUNET_CONTAINER_Heap *heap, void *element, GNUNET_CONTAINER_HeapCostType cost)
Inserts a new element into the heap.
void * GNUNET_CONTAINER_heap_peek(const struct GNUNET_CONTAINER_Heap *heap)
Get element stored at the root of heap.
unsigned int GNUNET_CONTAINER_heap_get_size(const struct GNUNET_CONTAINER_Heap *heap)
Get the current size of the heap.
#define GNUNET_assert(cond)
Use this for fatal errors that cannot be handled.
const char * GNUNET_h2s(const struct GNUNET_HashCode *hc)
Convert a hash value to a string (for printing debug messages).
@ GNUNET_ERROR_TYPE_DEBUG
#define GNUNET_malloc(size)
Wrapper around malloc.
#define GNUNET_new_array(n, type)
Allocate a size n array with structs or unions of the given type.
void GNUNET_STATISTICS_update(struct GNUNET_STATISTICS_Handle *handle, const char *name, int64_t delta, int make_persistent)
Set statistic value for the peer.
#define GNUNET_TIME_UNIT_SECONDS
One second.
struct GNUNET_TIME_Absolute GNUNET_TIME_absolute_get(void)
Get the current time.
Definition: time.c:110
struct GNUNET_TIME_Absolute GNUNET_TIME_absolute_subtract(struct GNUNET_TIME_Absolute start, struct GNUNET_TIME_Relative duration)
Subtract a given relative duration from the given start time.
Definition: time.c:468
struct GNUNET_TIME_Absolute GNUNET_TIME_relative_to_absolute(struct GNUNET_TIME_Relative rel)
Convert relative time to an absolute time in the future.
Definition: time.c:315
struct GNUNET_TIME_Relative GNUNET_TIME_relative_multiply(struct GNUNET_TIME_Relative rel, unsigned long long factor)
Multiply relative time by a given factor.
Definition: time.c:483
#define GNUNET_TIME_UNIT_FOREVER_ABS
Constant used to specify "forever".
A 512-bit hashcode.
The identity of the host (wraps the signing key of the peer).
uint64_t abs_value_us
The actual value.
const struct GNUNET_PeerIdentity * target
Identity of a peer hosting the content, otherwise NULl.
struct GNUNET_TIME_Absolute ttl
Current TTL for the request.
enum GNUNET_BLOCK_Type type
Type of the requested block.
enum GSF_PendingRequestOptions options
Options for the request.
uint32_t anonymity_level
Desired anonymity level.
struct GNUNET_HashCode query
Primary query hash for this request.
struct GNUNET_TIME_Absolute start_time
When did we start with the request.
uint32_t priority
Priority that this request (still) has for us.
uint32_t original_priority
Priority that this request (originally) had for us.
An active request.
unsigned int replies_seen_size
Length of the 'replies_seen' array.
GNUNET_PEER_Id sender_pid
Identity of the peer that we should use for the 'sender' (recipient of the response) when forwarding ...
GSF_PendingRequestReplyHandler rh
Function to call if we encounter a reply.
void * rh_cls
Closure for rh.
struct GSF_PendingRequestData public_data
Public data for the request.
struct GNUNET_CONTAINER_HeapNode * hnode
Entry for this pending request in the expiration heap, or NULL.
GNUNET_PEER_Id origin_pid
Identity of the peer that we should never forward this query to since it originated this query (0 for...

References GNUNET_TIME_Absolute::abs_value_us, anonymity_level, GSF_PendingRequestData::anonymity_level, GSF_PendingRequest::bg, gettext_noop, GNUNET_assert, GNUNET_BLOCK_group_create(), GNUNET_BLOCK_REPLY_TYPE_NOT_SUPPORTED, GNUNET_BLOCK_TYPE_ANY, GNUNET_BLOCK_TYPE_FS_UBLOCK, GNUNET_CONTAINER_heap_get_size(), GNUNET_CONTAINER_heap_insert(), GNUNET_CONTAINER_heap_peek(), GNUNET_CONTAINER_multihashmap_put(), GNUNET_CONTAINER_MULTIHASHMAPOPTION_MULTIPLE, GNUNET_ERROR_TYPE_DEBUG, GNUNET_h2s(), GNUNET_log, GNUNET_malloc, GNUNET_memcpy, GNUNET_new_array, GNUNET_NO, GNUNET_STATISTICS_update(), GNUNET_TIME_absolute_get(), GNUNET_TIME_absolute_subtract(), GNUNET_TIME_relative_multiply(), GNUNET_TIME_relative_to_absolute(), GNUNET_TIME_UNIT_FOREVER_ABS, GNUNET_TIME_UNIT_SECONDS, GNUNET_YES, GSF_block_ctx, GSF_pending_request_cancel_(), GSF_PRO_BLOOMFILTER_FULL_REFRESH, GSF_PRO_FORWARD_ONLY, GSF_PRO_REQUEST_NEVER_EXPIRES, GSF_stats, GSF_PendingRequest::hnode, max_pending_requests, options, GSF_PendingRequestData::options, GSF_PendingRequest::origin_pid, GSF_PendingRequestData::original_priority, pr_map, GSF_PendingRequestData::priority, GSF_PendingRequest::public_data, GSF_PendingRequestData::query, refresh_bloomfilter(), GSF_PendingRequest::replies_seen, GSF_PendingRequest::replies_seen_count, GSF_PendingRequest::replies_seen_size, requests_by_expiration_heap, GSF_PendingRequest::rh, GSF_PendingRequest::rh_cls, GSF_PendingRequest::sender_pid, GSF_PendingRequestData::start_time, GSF_PendingRequestData::target, GSF_PendingRequestData::ttl, GSF_PendingRequestData::type, and type.

Referenced by handle_client_start_search(), and handle_p2p_get().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_pending_request_get_data_()

struct GSF_PendingRequestData* GSF_pending_request_get_data_ ( struct GSF_PendingRequest pr)

Obtain the public data associated with a pending request.

Parameters
prpending request
Returns
associated public data

Definition at line 436 of file gnunet-service-fs_pr.c.

437 {
438  return &pr->public_data;
439 }

References GSF_PendingRequest::public_data.

Referenced by client_response_handler(), free_pending_request(), get_latest(), get_rp_key(), GSF_plan_add_(), GSF_plan_notify_peer_disconnect_(), GSF_plan_notify_request_done_(), handle_client_start_search(), handle_p2p_get(), handle_p2p_reply(), merge_pr(), peer_request_destroy(), plan(), start_p2p_processing(), and test_exist_cb().

Here is the caller graph for this function:

◆ GSF_pending_request_is_compatible_()

int GSF_pending_request_is_compatible_ ( struct GSF_PendingRequest pra,
struct GSF_PendingRequest prb 
)

Test if two pending requests are compatible (would generate the same query modulo filters and should thus be processed jointly).

Parameters
praa pending request
prbanother pending request
Returns
GNUNET_OK if the requests are compatible

Definition at line 452 of file gnunet-service-fs_pr.c.

454 {
455  if ((pra->public_data.type != prb->public_data.type) ||
456  (0 != memcmp (&pra->public_data.query,
457  &prb->public_data.query,
458  sizeof(struct GNUNET_HashCode))))
459  return GNUNET_NO;
460  return GNUNET_OK;
461 }

References GNUNET_NO, GNUNET_OK, GSF_PendingRequest::public_data, GSF_PendingRequestData::query, and GSF_PendingRequestData::type.

Referenced by merge_pr().

Here is the caller graph for this function:

◆ GSF_pending_request_update_()

void GSF_pending_request_update_ ( struct GSF_PendingRequest pr,
const struct GNUNET_HashCode replies_seen,
unsigned int  replies_seen_count 
)

Update a given pending request with additional replies that have been seen.

Parameters
prrequest to update
replies_seenhash codes of replies that we've seen
replies_seen_countsize of the replies_seen array

Definition at line 473 of file gnunet-service-fs_pr.c.

476 {
477  if (replies_seen_count + pr->replies_seen_count < pr->replies_seen_count)
478  return; /* integer overflow */
480  {
481  /* we're responsible for the BF, full refresh */
482  if (replies_seen_count + pr->replies_seen_count > pr->replies_seen_size)
484  pr->replies_seen_size,
485  replies_seen_count + pr->replies_seen_count);
487  replies_seen,
488  sizeof(struct GNUNET_HashCode) * replies_seen_count);
489  pr->replies_seen_count += replies_seen_count;
491  }
492  else
493  {
494  if (NULL == pr->bg)
495  {
496  /* we're not the initiator, but the initiator did not give us
497  * any bloom-filter, so we need to create one on-the-fly */
499  }
500  else
501  {
504  replies_seen,
505  pr->replies_seen_count));
506  }
507  }
508  if (NULL != pr->gh)
510  replies_seen_count,
511  replies_seen);
512 }
void GNUNET_DHT_get_filter_known_results(struct GNUNET_DHT_GetHandle *get_handle, unsigned int num_results, const struct GNUNET_HashCode *results)
Tell the DHT not to return any of the following known results to this client.
Definition: dht_api.c:1163
#define GNUNET_array_grow(arr, size, tsize)
Grow a well-typed (!) array.
struct GNUNET_DHT_GetHandle * gh
DHT request handle for this request (or NULL for none).

References GSF_PendingRequest::bg, GSF_PendingRequest::gh, GNUNET_array_grow, GNUNET_BLOCK_group_set_seen(), GNUNET_break, GNUNET_DHT_get_filter_known_results(), GNUNET_memcpy, GNUNET_OK, GSF_PRO_BLOOMFILTER_FULL_REFRESH, GSF_PendingRequestData::options, GSF_PendingRequest::public_data, refresh_bloomfilter(), GSF_PendingRequest::replies_seen, GSF_PendingRequest::replies_seen_count, GSF_PendingRequest::replies_seen_size, and GSF_PendingRequestData::type.

Referenced by handle_client_start_search().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_pending_request_get_message_()

struct GNUNET_MQ_Envelope* GSF_pending_request_get_message_ ( struct GSF_PendingRequest pr)

Generate the message corresponding to the given pending request for transmission to other peers.

Parameters
prrequest to generate the message for
Returns
envelope with the request message

Definition at line 523 of file gnunet-service-fs_pr.c.

524 {
525  struct GNUNET_MQ_Envelope *env;
526  struct GetMessage *gm;
527  struct GNUNET_PeerIdentity *ext;
528  unsigned int k;
529  uint32_t bm;
530  uint32_t prio;
531  size_t bf_size;
532  struct GNUNET_TIME_Absolute now;
533  int64_t ttl;
534  int do_route;
535  void *bf_data;
536  uint32_t bf_nonce;
537 
539  "Building request message for `%s' of type %d\n",
541  pr->public_data.type);
542  k = 0;
543  bm = 0;
544  do_route = (0 == (pr->public_data.options & GSF_PRO_FORWARD_ONLY));
545  if ((! do_route) && (pr->sender_pid == 0))
546  {
547  GNUNET_break (0);
548  do_route = GNUNET_YES;
549  }
550  if (! do_route)
551  {
553  k++;
554  }
555  if (NULL != pr->public_data.target)
556  {
558  k++;
559  }
560  if (GNUNET_OK !=
562  &bf_nonce,
563  &bf_data,
564  &bf_size))
565  {
566  bf_size = 0;
567  bf_data = NULL;
568  }
569  env = GNUNET_MQ_msg_extra (gm,
570  bf_size + k * sizeof(struct GNUNET_PeerIdentity),
572  gm->type = htonl (pr->public_data.type);
573  if (do_route)
575  pr->public_data.priority + 1);
576  else
577  prio = 0;
578  pr->public_data.priority -= prio;
580  pr->public_data.respect_offered += prio;
581  gm->priority = htonl (prio);
582  now = GNUNET_TIME_absolute_get ();
583  ttl = (int64_t) (pr->public_data.ttl.abs_value_us - now.abs_value_us);
584  gm->ttl = htonl (ttl / 1000LL / 1000LL);
585  gm->filter_mutator = htonl (bf_nonce);
586  gm->hash_bitmap = htonl (bm);
587  gm->query = pr->public_data.query;
588  ext = (struct GNUNET_PeerIdentity *) &gm[1];
589  k = 0;
590  if (! do_route)
591  GNUNET_PEER_resolve (pr->sender_pid, &ext[k++]);
592  if (NULL != pr->public_data.target)
593  ext[k++] = *pr->public_data.target;
594  GNUNET_memcpy (&ext[k], bf_data, bf_size);
595  GNUNET_free (bf_data);
596  return env;
597 }
struct GNUNET_MQ_Envelope * env
Definition: 005.c:1
#define GET_MESSAGE_BIT_TRANSMIT_TO
The peer identity of a peer that had claimed to have the content previously is included (can be used ...
#define GET_MESSAGE_BIT_RETURN_TO
The peer identity of a peer waiting for the reply is included (used if the response should be transmi...
enum GNUNET_GenericReturnValue GNUNET_BLOCK_group_serialize(struct GNUNET_BLOCK_Group *bg, uint32_t *nonce, void **raw_data, size_t *raw_data_size)
Serialize state of a block group.
Definition: block.c:153
#define GNUNET_free(ptr)
Wrapper around free.
#define GNUNET_MQ_msg_extra(mvar, esize, type)
Allocate an envelope, with extra space allocated after the space needed by the message struct.
Definition: gnunet_mq_lib.h:52
void GNUNET_PEER_resolve(GNUNET_PEER_Id id, struct GNUNET_PeerIdentity *pid)
Convert an interned PID to a normal peer identity.
Definition: peer.c:225
#define GNUNET_MESSAGE_TYPE_FS_GET
P2P request for content (one FS to another).
Time for absolute times used by GNUnet, in microseconds.
uint32_t respect_offered
How much respect did we (in total) offer for this request so far (estimate, because we might have the...
uint32_t num_transmissions
Counter for how often this request has been transmitted (estimate, because we might have the same req...
Message to the datastore service asking about specific content.
Definition: datastore.h:141
uint32_t type
Desired content type.
Definition: datastore.h:150
uint32_t priority
How important is this request (network byte order)
uint32_t filter_mutator
The content hash should be mutated using this value before checking against the bloomfilter (used to ...
struct GNUNET_HashCode query
Hashcodes of the file(s) we're looking for.
int32_t ttl
Relative time to live in MILLISECONDS (network byte order)
uint32_t hash_bitmap
Which of the optional hash codes are present at the end of the message? See GET_MESSAGE_BIT_xx consta...

References GNUNET_TIME_Absolute::abs_value_us, GSF_PendingRequest::bg, env, GetMessage::filter_mutator, GET_MESSAGE_BIT_RETURN_TO, GET_MESSAGE_BIT_TRANSMIT_TO, GNUNET_BLOCK_group_serialize(), GNUNET_break, GNUNET_CRYPTO_QUALITY_WEAK, GNUNET_CRYPTO_random_u32(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_free, GNUNET_h2s(), GNUNET_log, GNUNET_memcpy, GNUNET_MESSAGE_TYPE_FS_GET, GNUNET_MQ_msg_extra, GNUNET_OK, GNUNET_PEER_resolve(), GNUNET_TIME_absolute_get(), GNUNET_YES, GSF_PRO_FORWARD_ONLY, GetMessage::hash_bitmap, GSF_PendingRequestData::num_transmissions, GSF_PendingRequestData::options, GetMessage::priority, GSF_PendingRequestData::priority, GSF_PendingRequest::public_data, GetMessage::query, GSF_PendingRequestData::query, GSF_PendingRequestData::respect_offered, GSF_PendingRequest::sender_pid, GSF_PendingRequestData::target, GetMessage::ttl, GSF_PendingRequestData::ttl, GetMessage::type, and GSF_PendingRequestData::type.

Referenced by schedule_peer_transmission().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ clean_request()

static int clean_request ( void *  cls,
const struct GNUNET_HashCode key,
void *  value 
)
static

Iterator to free pending requests.

Parameters
clsclosure, unused
keycurrent key code
valuevalue in the hash map (pending request)
Returns
GNUNET_YES (we should continue to iterate)

Definition at line 609 of file gnunet-service-fs_pr.c.

610 {
611  struct GSF_PendingRequest *pr = value;
613 
615  "Cleaning up pending request for `%s'.\n",
616  GNUNET_h2s (key));
617  if (NULL != pr->cadet_request)
618  {
621  pr->cadet_request = NULL;
622  }
623  if (NULL != (cont = pr->llc_cont))
624  {
625  pr->llc_cont = NULL;
626  cont (pr->llc_cont_cls,
627  pr,
628  pr->local_result);
629  }
633  pr->bg = NULL;
635  pr->sender_pid = 0;
637  pr->origin_pid = 0;
638  if (NULL != pr->hnode)
639  {
641  pr->hnode = NULL;
642  }
643  if (NULL != pr->qe)
644  {
646  pr->qe = NULL;
647  }
648  if (NULL != pr->gh)
649  {
650  GNUNET_DHT_get_stop (pr->gh);
651  pr->gh = NULL;
652  }
653  if (NULL != pr->warn_task)
654  {
656  pr->warn_task = NULL;
657  }
658  GNUNET_assert (
659  GNUNET_OK ==
662  gettext_noop ("# Pending requests active"),
663  -1,
664  GNUNET_NO);
665  GNUNET_free (pr);
666  return GNUNET_YES;
667 }
struct GNUNET_HashCode key
The key used in the DHT.
static char * value
Value of the record to add/remove.
void GSF_cadet_query_cancel(struct GSF_CadetRequest *sr)
Cancel an active request; must not be called after 'proc' was called.
void GSF_plan_notify_request_done_(struct GSF_PendingRequest *pr)
Notify the plan about a request being done; destroy all entries associated with this request.
#define CADET_RETRY_MAX
If obtaining a block via cadet fails, how often do we retry it before giving up for good (and stickin...
void(* GSF_LocalLookupContinuation)(void *cls, struct GSF_PendingRequest *pr, enum GNUNET_BLOCK_ReplyEvaluationResult result)
Function to be called after we're done processing replies from the local lookup.
void GNUNET_DATASTORE_cancel(struct GNUNET_DATASTORE_QueueEntry *qe)
Cancel a datastore operation.
void GNUNET_DHT_get_stop(struct GNUNET_DHT_GetHandle *get_handle)
Stop async DHT-get.
Definition: dht_api.c:1187
int GNUNET_CONTAINER_multihashmap_remove(struct GNUNET_CONTAINER_MultiHashMap *map, const struct GNUNET_HashCode *key, const void *value)
Remove the given key-value pair from the map.
void * GNUNET_CONTAINER_heap_remove_node(struct GNUNET_CONTAINER_HeapNode *node)
Removes a node from the heap.
void GNUNET_PEER_change_rc(GNUNET_PEER_Id id, int delta)
Change the reference counter of an interned PID.
Definition: peer.c:197
void * GNUNET_SCHEDULER_cancel(struct GNUNET_SCHEDULER_Task *task)
Cancel the task with the specified identifier.
Definition: scheduler.c:957
enum GNUNET_BLOCK_ReplyEvaluationResult local_result
Last result from the local datastore lookup evaluation.
struct GNUNET_DATASTORE_QueueEntry * qe
Datastore queue entry for this request (or NULL for none).
void * llc_cont_cls
Closure for llc_cont.
unsigned int cadet_retry_count
How often have we retried this request via 'cadet'? (used to bound overall retries).
struct GSF_CadetRequest * cadet_request
Cadet request handle for this request (or NULL for none).
GSF_LocalLookupContinuation llc_cont
Function to call upon completion of the local get request, or NULL for none.
struct GNUNET_SCHEDULER_Task * warn_task
Task that warns us if the local datastore lookup takes too long.

References GSF_PendingRequest::bg, GSF_PendingRequest::cadet_request, GSF_PendingRequest::cadet_retry_count, CADET_RETRY_MAX, gettext_noop, GSF_PendingRequest::gh, GNUNET_assert, GNUNET_BLOCK_group_destroy(), GNUNET_CONTAINER_heap_remove_node(), GNUNET_CONTAINER_multihashmap_remove(), GNUNET_DATASTORE_cancel(), GNUNET_DHT_get_stop(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_free, GNUNET_h2s(), GNUNET_log, GNUNET_NO, GNUNET_OK, GNUNET_PEER_change_rc(), GNUNET_SCHEDULER_cancel(), GNUNET_STATISTICS_update(), GNUNET_YES, GSF_cadet_query_cancel(), GSF_plan_notify_request_done_(), GSF_stats, GSF_PendingRequest::hnode, key, GSF_PendingRequest::llc_cont, GSF_PendingRequest::llc_cont_cls, GSF_PendingRequest::local_result, GSF_PendingRequest::origin_pid, pr_map, GSF_PendingRequest::public_data, GSF_PendingRequest::qe, GSF_PendingRequestData::query, GSF_PendingRequest::replies_seen, GSF_PendingRequest::sender_pid, value, and GSF_PendingRequest::warn_task.

Referenced by GSF_pending_request_cancel_().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_pending_request_cancel_()

void GSF_pending_request_cancel_ ( struct GSF_PendingRequest pr,
int  full_cleanup 
)

Explicitly cancel a pending request.

Parameters
prrequest to cancel
full_cleanupfully purge the request

Definition at line 677 of file gnunet-service-fs_pr.c.

678 {
680 
681  if (NULL == pr_map)
682  return; /* already cleaned up! */
683  if (GNUNET_NO == full_cleanup)
684  {
685  /* make request inactive (we're no longer interested in more results),
686  * but do NOT remove from our data-structures, we still need it there
687  * to prevent the request from looping */
688  pr->rh = NULL;
689  if (NULL != pr->cadet_request)
690  {
693  pr->cadet_request = NULL;
694  }
695  if (NULL != (cont = pr->llc_cont))
696  {
697  pr->llc_cont = NULL;
698  cont (pr->llc_cont_cls,
699  pr,
700  pr->local_result);
701  }
703  if (NULL != pr->qe)
704  {
706  pr->qe = NULL;
707  }
708  if (NULL != pr->gh)
709  {
710  GNUNET_DHT_get_stop (pr->gh);
711  pr->gh = NULL;
712  }
713  if (NULL != pr->warn_task)
714  {
716  pr->warn_task = NULL;
717  }
718  return;
719  }
721  clean_request (NULL, &pr->public_data.query, pr));
722 }
static int clean_request(void *cls, const struct GNUNET_HashCode *key, void *value)
Iterator to free pending requests.

References GSF_PendingRequest::cadet_request, GSF_PendingRequest::cadet_retry_count, CADET_RETRY_MAX, clean_request(), GSF_PendingRequest::gh, GNUNET_assert, GNUNET_DATASTORE_cancel(), GNUNET_DHT_get_stop(), GNUNET_NO, GNUNET_SCHEDULER_cancel(), GNUNET_YES, GSF_cadet_query_cancel(), GSF_plan_notify_request_done_(), GSF_PendingRequest::llc_cont, GSF_PendingRequest::llc_cont_cls, GSF_PendingRequest::local_result, pr_map, GSF_PendingRequest::public_data, GSF_PendingRequest::qe, GSF_PendingRequestData::query, GSF_PendingRequest::rh, and GSF_PendingRequest::warn_task.

Referenced by cancel_pending_request(), client_request_destroy(), GSF_pending_request_create_(), and test_exist_cb().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_iterate_pending_requests_()

void GSF_iterate_pending_requests_ ( GSF_PendingRequestIterator  it,
void *  cls 
)

Iterate over all pending requests.

Parameters
itfunction to call for each request
clsclosure for it

Definition at line 732 of file gnunet-service-fs_pr.c.

733 {
735  pr_map,
737  cls);
738 }
int GNUNET_CONTAINER_multihashmap_iterate(struct GNUNET_CONTAINER_MultiHashMap *map, GNUNET_CONTAINER_MulitHashMapIteratorCallback it, void *it_cls)
Iterate over all entries in the map.
int(* GNUNET_CONTAINER_MulitHashMapIteratorCallback)(void *cls, const struct GNUNET_HashCode *key, void *value)
Iterator over hash map entries.

References GNUNET_CONTAINER_multihashmap_iterate(), and pr_map.

Referenced by GSF_peer_connect_handler().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ update_request_performance_data()

static void update_request_performance_data ( struct ProcessReplyClosure prq,
struct GSF_PendingRequest pr 
)
static

Update the performance data for the sender (if any) since the sender successfully answered one of our queries.

Parameters
prqinformation about the sender
prrequest that was satisfied

Definition at line 801 of file gnunet-service-fs_pr.c.

803 {
804  if (prq->sender == NULL)
805  return;
808  prq->priority);
809 }
void GSF_peer_update_performance_(struct GSF_ConnectedPeer *cp, struct GNUNET_TIME_Absolute request_time, uint32_t request_priority)
Report on receiving a reply; update the performance record of the given peer.
uint32_t priority
How much was this reply worth to us?
struct GSF_ConnectedPeer * sender
Who gave us this reply? NULL for local host (or DHT)

◆ process_reply()

static enum GNUNET_GenericReturnValue process_reply ( void *  cls,
const struct GNUNET_HashCode key,
void *  value 
)
static

We have received a reply; handle it!

Parameters
clsresponse (a struct ProcessReplyClosure)
keyour query
valuevalue in the hash map (info about the query)
Returns
GNUNET_YES (we should continue to iterate)

Definition at line 801 of file gnunet-service-fs_pr.c.

824 {
825  struct ProcessReplyClosure *prq = cls;
826  struct GSF_PendingRequest *pr = value;
827  struct GNUNET_HashCode chash;
829 
830  if (NULL == pr->rh)
831  return GNUNET_YES;
833  "Matched result (type %u) for query `%s' with pending request\n",
834  (unsigned int) prq->type,
835  GNUNET_h2s (key));
837  gettext_noop ("# replies received and matched"),
838  1,
839  GNUNET_NO);
841  prq->type,
842  pr->bg,
843  key,
844  NULL, 0,
845  prq->data,
846  prq->size);
847  switch (prq->eval)
848  {
851  break;
853  /* short cut: stop processing early, no BF-update, etc. */
858  .rel_value_us);
859  if (GNUNET_YES !=
861  .pr_head,
862  prq->sender,
865  /* pass on to other peers / local clients */
866  pr->rh (pr->rh_cls,
867  prq->eval,
868  pr,
869  prq->anonymity_level,
870  prq->expiration,
872  prq->type,
873  prq->data,
874  prq->size);
875  return GNUNET_YES;
877 #if INSANE_STATISTICS
879  "# duplicate replies discarded (bloomfilter)",
880  1,
881  GNUNET_NO);
882 #endif
884  "Duplicate response, discarding.\n");
885  return GNUNET_YES; /* duplicate */
888  "# irrelevant replies discarded",
889  1,
890  GNUNET_NO);
892  "Irrelevant response, ignoring.\n");
893  return GNUNET_YES;
895  GNUNET_break (0); /* bad installation? */
896  return GNUNET_NO;
897  }
898  /* update bloomfilter */
899  GNUNET_CRYPTO_hash (prq->data,
900  prq->size,
901  &chash);
903  &chash,
904  1);
905  if (NULL == prq->sender)
906  {
908  "Found result for query `%s' in local datastore\n",
909  GNUNET_h2s (key));
911  gettext_noop ("# results found locally"),
912  1,
913  GNUNET_NO);
914  }
915  else
916  {
917  GSF_dht_lookup_ (pr);
918  }
920  pr->public_data.priority = 0;
923  prq->request_found = GNUNET_YES;
924  /* finally, pass on to other peer / local client */
926  .pr_head,
927  prq->sender,
930  pr->rh (pr->rh_cls,
931  prq->eval,
932  pr,
933  prq->anonymity_level,
934  prq->expiration,
936  prq->type,
937  prq->data,
938  prq->size);
939  return GNUNET_YES;
940 }
struct GNUNET_LOAD_Value * GSF_rt_entry_lifetime
How long do requests typically stay in the routing table?
int GSF_request_plan_reference_get_last_transmission_(struct GSF_PendingRequestPlanBijection *pr_head, struct GSF_ConnectedPeer *sender, struct GNUNET_TIME_Absolute *result)
Get the last transmission attempt time for the request plan list referenced by pr_head,...
static void update_request_performance_data(struct ProcessReplyClosure *prq, struct GSF_PendingRequest *pr)
Update the performance data for the sender (if any) since the sender successfully answered one of our...
void GSF_dht_lookup_(struct GSF_PendingRequest *pr)
Consider looking up the data in the DHT (anonymity-level permitting).
void GSF_pending_request_update_(struct GSF_PendingRequest *pr, const struct GNUNET_HashCode *replies_seen, unsigned int replies_seen_count)
Update a given pending request with additional replies that have been seen.
enum GNUNET_BLOCK_ReplyEvaluationResult GNUNET_BLOCK_check_reply(struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, struct GNUNET_BLOCK_Group *group, const struct GNUNET_HashCode *query, const void *xquery, size_t xquery_size, const void *reply_block, size_t reply_block_size)
Function called to validate if a reply is good for a particular query.
Definition: block.c:317
@ GNUNET_BLOCK_REPLY_OK_MORE
Valid result, and there may be more.
@ GNUNET_BLOCK_REPLY_OK_DUPLICATE
Valid result, but suppressed because it is a duplicate.
@ GNUNET_BLOCK_REPLY_OK_LAST
Last possible valid result.
@ GNUNET_BLOCK_REPLY_IRRELEVANT
Block does not match xquery (valid result, not relevant for the request)
void GNUNET_CRYPTO_hash(const void *block, size_t size, struct GNUNET_HashCode *ret)
Compute hash of a given block.
Definition: crypto_hash.c:41
void GNUNET_LOAD_update(struct GNUNET_LOAD_Value *load, uint64_t data)
Update the current load.
Definition: load.c:236
struct GNUNET_TIME_Relative GNUNET_TIME_absolute_get_duration(struct GNUNET_TIME_Absolute whence)
Get the duration of an operation as the difference of the current time and the given start time "henc...
Definition: time.c:435
static GNUNET_CronTime last_transmission
uint64_t rel_value_us
The actual value.
struct GSF_PendingRequestPlanBijection * pr_head
Fields for the plan module to track a DLL with the request.
unsigned int results_found
Number of results we have found for this request so far.
Closure for process_reply() function.
size_t size
Size of data.
struct GNUNET_TIME_Absolute expiration
When the reply expires.
enum GNUNET_BLOCK_ReplyEvaluationResult eval
Evaluation result (returned).
const void * data
The data for the reply.
int request_found
Did we find a matching request?
enum GNUNET_BLOCK_Type type
Type of the block.
uint32_t anonymity_level
Anonymity requirements for this reply.

References GSF_peer_update_performance_(), ProcessReplyClosure::priority, GSF_PendingRequest::public_data, ProcessReplyClosure::sender, and GSF_PendingRequestData::start_time.

Referenced by cadet_reply_proc(), handle_dht_reply(), handle_p2p_put(), and process_local_reply().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ put_migration_continuation()

static void put_migration_continuation ( void *  cls,
int  success,
struct GNUNET_TIME_Absolute  min_expiration,
const char *  msg 
)
static

Continuation called to notify client about result of the operation.

Parameters
clsclosure
successGNUNET_SYSERR on failure
min_expirationminimum expiration time required for content to be stored
msgNULL on success, otherwise an error message

Definition at line 976 of file gnunet-service-fs_pr.c.

980 {
981  struct PutMigrationContext *pmc = cls;
982  struct GSF_ConnectedPeer *cp;
983  struct GNUNET_TIME_Relative mig_pause;
984  struct GSF_PeerPerformanceData *ppd;
985 
986  if (NULL != datastore_put_load)
987  {
988  if (GNUNET_SYSERR != success)
989  {
992  .rel_value_us);
993  }
994  else
995  {
996  /* on queue failure / timeout, increase the put load dramatically */
998  GNUNET_TIME_UNIT_MINUTES.rel_value_us);
999  }
1000  }
1001  cp = GSF_peer_get_ (&pmc->origin);
1002  if (GNUNET_OK == success)
1003  {
1004  if (NULL != cp)
1005  {
1006  ppd = GSF_get_peer_performance_data_ (cp);
1007  ppd->migration_delay.rel_value_us /= 2;
1008  }
1009  GNUNET_free (pmc);
1010  return;
1011  }
1012  if ((GNUNET_NO == success) && (GNUNET_NO == pmc->requested) && (NULL != cp))
1013  {
1014  ppd = GSF_get_peer_performance_data_ (cp);
1015  if (min_expiration.abs_value_us > 0)
1016  {
1018  "Asking to stop migration for %s because datastore is full\n",
1021  GNUNET_YES));
1023  }
1024  else
1025  {
1027  ppd->migration_delay);
1028  ppd->migration_delay =
1030  mig_pause.rel_value_us =
1033  ppd->migration_delay =
1035  GNUNET_log (
1037  "Replicated content already exists locally, asking to stop migration for %s\n",
1040  GNUNET_TIME_relative_to_absolute (mig_pause));
1041  }
1042  }
1043  GNUNET_free (pmc);
1045  gettext_noop ("# Datastore `PUT' failures"),
1046  1,
1047  GNUNET_NO);
1048 }
static struct GNUNET_TIME_Absolute min_expiration
Minimum time that content should have to not be discarded instantly (time stamp of any content that w...
struct GSF_ConnectedPeer * GSF_peer_get_(const struct GNUNET_PeerIdentity *peer)
Get a handle for a connected peer.
void GSF_block_peer_migration_(struct GSF_ConnectedPeer *cp, struct GNUNET_TIME_Absolute block_time)
Ask a peer to stop migrating data to us until the given point in time.
struct GSF_PeerPerformanceData * GSF_get_peer_performance_data_(struct GSF_ConnectedPeer *cp)
Return the performance data record for the given peer.
static struct GNUNET_LOAD_Value * datastore_put_load
Datastore 'PUT' load tracking.
@ GNUNET_SYSERR
Definition: gnunet_common.h:93
uint64_t GNUNET_CRYPTO_random_u64(enum GNUNET_CRYPTO_Quality mode, uint64_t max)
Random on unsigned 64-bit values.
struct GNUNET_TIME_Relative GNUNET_TIME_relative_min(struct GNUNET_TIME_Relative t1, struct GNUNET_TIME_Relative t2)
Return the minimum of two relative time values.
Definition: time.c:342
struct GNUNET_TIME_Relative GNUNET_TIME_relative_saturating_multiply(struct GNUNET_TIME_Relative rel, unsigned long long factor)
Saturating multiply relative time by a given factor.
Definition: time.c:530
struct GNUNET_TIME_Relative GNUNET_TIME_relative_max(struct GNUNET_TIME_Relative t1, struct GNUNET_TIME_Relative t2)
Return the maximum of two relative time values.
Definition: time.c:350
#define GNUNET_TIME_UNIT_HOURS
One hour.
struct GNUNET_TIME_Relative GNUNET_TIME_absolute_get_remaining(struct GNUNET_TIME_Absolute future)
Given a timestamp in the future, how much time remains until then?
Definition: time.c:404
#define GNUNET_TIME_UNIT_MINUTES
One minute.
const char * GNUNET_STRINGS_relative_time_to_string(struct GNUNET_TIME_Relative delta, int do_round)
Give relative time in human-readable fancy format.
Definition: strings.c:570
Time for relative time used by GNUnet, in microseconds.
A connected peer.
Performance data kept for a peer.
struct GNUNET_TIME_Relative migration_delay
If we get content we already have from this peer, for how long do we block it? Adjusted based on the ...
Context for put_migration_continuation().
struct GNUNET_PeerIdentity origin
Request origin.
struct GNUNET_TIME_Absolute start
Start time for the operation.
int requested
GNUNET_YES if we had a matching request for this block, GNUNET_NO if not.

References GNUNET_TIME_Absolute::abs_value_us, datastore_put_load, gettext_noop, GNUNET_CRYPTO_QUALITY_WEAK, GNUNET_CRYPTO_random_u64(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_free, GNUNET_LOAD_update(), GNUNET_log, GNUNET_NO, GNUNET_OK, GNUNET_STATISTICS_update(), GNUNET_STRINGS_relative_time_to_string(), GNUNET_SYSERR, GNUNET_TIME_absolute_get_duration(), GNUNET_TIME_absolute_get_remaining(), GNUNET_TIME_relative_max(), GNUNET_TIME_relative_min(), GNUNET_TIME_relative_saturating_multiply(), GNUNET_TIME_relative_to_absolute(), GNUNET_TIME_UNIT_HOURS, GNUNET_TIME_UNIT_MINUTES, GNUNET_TIME_UNIT_SECONDS, GNUNET_YES, GSF_block_peer_migration_(), GSF_get_peer_performance_data_(), GSF_peer_get_(), GSF_stats, GSF_PeerPerformanceData::migration_delay, min_expiration, PutMigrationContext::origin, GNUNET_TIME_Relative::rel_value_us, PutMigrationContext::requested, and PutMigrationContext::start.

Referenced by handle_dht_reply(), and handle_p2p_put().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ test_put_load_too_high()

static int test_put_load_too_high ( uint32_t  priority)
static

Test if the DATABASE (PUT) load on this peer is too high to even consider processing the query at all.

Parameters
prioritythe priority of the item
Returns
GNUNET_YES if the load is too high to do anything (load high) GNUNET_NO to process normally (load normal or low)

Definition at line 1061 of file gnunet-service-fs_pr.c.

1062 {
1063  double ld;
1064 
1065  if (NULL == datastore_put_load)
1066  return GNUNET_NO;
1068  return GNUNET_NO; /* very fast */
1070  if (ld < 2.0 * (1 + priority))
1071  return GNUNET_NO;
1073  gettext_noop (
1074  "# storage requests dropped due to high load"),
1075  1,
1076  GNUNET_NO);
1077  return GNUNET_YES;
1078 }
double GNUNET_LOAD_get_average(struct GNUNET_LOAD_Value *load)
Get the average value given to update so far.
Definition: load.c:215
double GNUNET_LOAD_get_load(struct GNUNET_LOAD_Value *load)
Get the current load.
Definition: load.c:200

References datastore_put_load, gettext_noop, GNUNET_LOAD_get_average(), GNUNET_LOAD_get_load(), GNUNET_NO, GNUNET_STATISTICS_update(), GNUNET_YES, and GSF_stats.

Referenced by handle_dht_reply(), and handle_p2p_put().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ handle_dht_reply()

static void handle_dht_reply ( void *  cls,
struct GNUNET_TIME_Absolute  exp,
const struct GNUNET_HashCode key,
const struct GNUNET_DHT_PathElement get_path,
unsigned int  get_path_length,
const struct GNUNET_DHT_PathElement put_path,
unsigned int  put_path_length,
enum GNUNET_BLOCK_Type  type,
size_t  size,
const void *  data 
)
static

Iterator called on each result obtained for a DHT operation that expects a reply.

Parameters
clsclosure
expwhen will this value expire
keykey of the result
get_pathpeers on reply path (or NULL if not recorded)
get_path_lengthnumber of entries in get_path
put_pathpeers on the PUT path (or NULL if not recorded)
put_path_lengthnumber of entries in get_path
typetype of the result
sizenumber of bytes in data
datapointer to the result data

Definition at line 1097 of file gnunet-service-fs_pr.c.

1107 {
1108  struct GSF_PendingRequest *pr = cls;
1109  struct ProcessReplyClosure prq;
1110  struct PutMigrationContext *pmc;
1111 
1113  gettext_noop ("# Replies received from DHT"),
1114  1,
1115  GNUNET_NO);
1116  memset (&prq, 0, sizeof(prq));
1117  prq.data = data;
1118  prq.expiration = exp;
1119  /* do not allow migrated content to live longer than 1 year */
1122  prq.expiration);
1123  prq.size = size;
1124  prq.type = type;
1125  process_reply (&prq,
1126  key,
1127  pr);
1128  if ((GNUNET_YES == active_to_migration) &&
1129  (GNUNET_NO == test_put_load_too_high (prq.priority)))
1130  {
1132  "Replicating result for query `%s' with priority %u\n",
1133  GNUNET_h2s (key),
1134  prq.priority);
1135  pmc = GNUNET_new (struct PutMigrationContext);
1136  pmc->start = GNUNET_TIME_absolute_get ();
1137  pmc->requested = GNUNET_YES;
1138  if (NULL == GNUNET_DATASTORE_put (GSF_dsh,
1139  0,
1140  key,
1141  size,
1142  data,
1143  type,
1144  prq.priority,
1145  1 /* anonymity */,
1146  0 /* replication */,
1147  exp,
1148  1 + prq.priority,
1151  pmc))
1152  {
1154  GNUNET_SYSERR,
1156  NULL);
1157  }
1158  }
1159 }
uint32_t data
The data value.
struct GNUNET_DATASTORE_Handle * GSF_dsh
Our connection to the datastore.
static void put_migration_continuation(void *cls, int success, struct GNUNET_TIME_Absolute min_expiration, const char *msg)
Continuation called to notify client about result of the operation.
#define MAX_DATASTORE_QUEUE
Maximum size of the datastore queue for P2P operations.
static int active_to_migration
Are we allowed to migrate content to this peer.
static int test_put_load_too_high(uint32_t priority)
Test if the DATABASE (PUT) load on this peer is too high to even consider processing the query at all...
static enum GNUNET_GenericReturnValue process_reply(void *cls, const struct GNUNET_HashCode *key, void *value)
We have received a reply; handle it!
struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_put(struct GNUNET_DATASTORE_Handle *h, uint32_t rid, const struct GNUNET_HashCode *key, size_t size, const void *data, enum GNUNET_BLOCK_Type type, uint32_t priority, uint32_t anonymity, uint32_t replication, struct GNUNET_TIME_Absolute expiration, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_ContinuationWithStatus cont, void *cont_cls)
Store an item in the datastore.
#define GNUNET_new(type)
Allocate a struct or union of the given type.
struct GNUNET_TIME_Absolute GNUNET_TIME_absolute_min(struct GNUNET_TIME_Absolute t1, struct GNUNET_TIME_Absolute t2)
Return the minimum of two absolute time values.
Definition: time.c:358
#define GNUNET_TIME_UNIT_ZERO_ABS
Absolute time zero.
#define GNUNET_TIME_UNIT_YEARS
One year (365 days).
static unsigned int size
Size of the "table".
Definition: peer.c:67

References active_to_migration, ProcessReplyClosure::data, data, ProcessReplyClosure::expiration, gettext_noop, GNUNET_DATASTORE_put(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_h2s(), GNUNET_log, GNUNET_new, GNUNET_NO, GNUNET_STATISTICS_update(), GNUNET_SYSERR, GNUNET_TIME_absolute_get(), GNUNET_TIME_absolute_min(), GNUNET_TIME_relative_to_absolute(), GNUNET_TIME_UNIT_YEARS, GNUNET_TIME_UNIT_ZERO_ABS, GNUNET_YES, GSF_dsh, GSF_stats, key, MAX_DATASTORE_QUEUE, ProcessReplyClosure::priority, process_reply(), put_migration_continuation(), PutMigrationContext::requested, ProcessReplyClosure::size, size, PutMigrationContext::start, test_put_load_too_high(), ProcessReplyClosure::type, and type.

Referenced by GSF_dht_lookup_().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_dht_lookup_()

void GSF_dht_lookup_ ( struct GSF_PendingRequest pr)

Consider looking up the data in the DHT (anonymity-level permitting).

Parameters
prthe pending request to process

Definition at line 1168 of file gnunet-service-fs_pr.c.

1169 {
1170  const void *xquery;
1171  size_t xquery_size;
1172  struct GNUNET_PeerIdentity pi;
1173  char buf[sizeof(struct GNUNET_HashCode) * 2] GNUNET_ALIGN;
1174 
1175  if (0 != pr->public_data.anonymity_level)
1176  return;
1177  if (NULL != pr->gh)
1178  {
1179  GNUNET_DHT_get_stop (pr->gh);
1180  pr->gh = NULL;
1181  }
1182  xquery = NULL;
1183  xquery_size = 0;
1184  if (0 != (pr->public_data.options & GSF_PRO_FORWARD_ONLY))
1185  {
1186  GNUNET_assert (0 != pr->sender_pid);
1188  GNUNET_memcpy (&buf[xquery_size], &pi, sizeof(struct GNUNET_PeerIdentity));
1189  xquery_size += sizeof(struct GNUNET_PeerIdentity);
1190  }
1192  pr->public_data.type,
1193  &pr->public_data.query,
1196  xquery,
1197  xquery_size,
1199  pr);
1200  if ((NULL != pr->gh) && (0 != pr->replies_seen_count))
1202  pr->replies_seen_count,
1203  pr->replies_seen);
1204 }
static struct GNUNET_PEERINFO_Handle * pi
Handle to peerinfo service.
struct GNUNET_DHT_Handle * GSF_dht
Handle for DHT operations.
#define DHT_GET_REPLICATION
Desired replication level for GETs.
static void handle_dht_reply(void *cls, struct GNUNET_TIME_Absolute exp, const struct GNUNET_HashCode *key, const struct GNUNET_DHT_PathElement *get_path, unsigned int get_path_length, const struct GNUNET_DHT_PathElement *put_path, unsigned int put_path_length, enum GNUNET_BLOCK_Type type, size_t size, const void *data)
Iterator called on each result obtained for a DHT operation that expects a reply.
static char buf[2048]
#define GNUNET_ALIGN
gcc-ism to force alignment; we use this to align char-arrays that may then be cast to 'struct's.
struct GNUNET_DHT_GetHandle * GNUNET_DHT_get_start(struct GNUNET_DHT_Handle *handle, enum GNUNET_BLOCK_Type type, const struct GNUNET_HashCode *key, uint32_t desired_replication_level, enum GNUNET_DHT_RouteOption options, const void *xquery, size_t xquery_size, GNUNET_DHT_GetIterator iter, void *iter_cls)
Perform an asynchronous GET operation on the DHT identified.
Definition: dht_api.c:1114
@ GNUNET_DHT_RO_DEMULTIPLEX_EVERYWHERE
Each peer along the way should process the request (otherwise only peers locally closest to the key w...

References GSF_PendingRequestData::anonymity_level, buf, DHT_GET_REPLICATION, GSF_PendingRequest::gh, GNUNET_ALIGN, GNUNET_assert, GNUNET_DHT_get_filter_known_results(), GNUNET_DHT_get_start(), GNUNET_DHT_get_stop(), GNUNET_DHT_RO_DEMULTIPLEX_EVERYWHERE, GNUNET_memcpy, GNUNET_PEER_resolve(), GSF_dht, GSF_PRO_FORWARD_ONLY, handle_dht_reply(), GSF_PendingRequestData::options, pi, GSF_PendingRequest::public_data, GSF_PendingRequestData::query, GSF_PendingRequest::replies_seen, GSF_PendingRequest::replies_seen_count, GSF_PendingRequest::sender_pid, and GSF_PendingRequestData::type.

Referenced by start_p2p_processing().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ cadet_reply_proc()

static void cadet_reply_proc ( void *  cls,
enum GNUNET_BLOCK_Type  type,
struct GNUNET_TIME_Absolute  expiration,
size_t  data_size,
const void *  data 
)
static

Function called with a reply from the cadet.

Parameters
clsthe pending request struct
typetype of the block, ANY on error
expirationexpiration time for the block
data_sizenumber of bytes in data, 0 on error
datareply block data, NULL on error

Definition at line 1217 of file gnunet-service-fs_pr.c.

1222 {
1223  struct GSF_PendingRequest *pr = cls;
1224  struct ProcessReplyClosure prq;
1225  struct GNUNET_HashCode query;
1226 
1227  pr->cadet_request = NULL;
1228  if (GNUNET_OK !=
1230  type,
1231  data,
1232  data_size))
1233  {
1234  GNUNET_break_op (0);
1235  return;
1236  }
1237  if (GNUNET_BLOCK_TYPE_ANY == type)
1238  {
1239  GNUNET_break (NULL == data);
1240  GNUNET_break (0 == data_size);
1241  pr->cadet_retry_count++;
1243  return; /* give up on cadet */
1244  GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "Error retrieiving block via cadet\n");
1245  /* retry -- without delay, as this is non-anonymous
1246  and cadet/cadet connect will take some time anyway */
1248  &pr->public_data.query,
1249  pr->public_data.type,
1251  pr);
1252  return;
1253  }
1254  if (GNUNET_YES !=
1256  type,
1257  data,
1258  data_size,
1259  &query))
1260  {
1262  "Failed to derive key for block of type %d\n",
1263  (int) type);
1264  GNUNET_break_op (0);
1265  return;
1266  }
1268  gettext_noop ("# Replies received from CADET"),
1269  1,
1270  GNUNET_NO);
1271  memset (&prq, 0, sizeof(prq));
1272  prq.data = data;
1273  prq.expiration = expiration;
1274  /* do not allow migrated content to live longer than 1 year */
1277  prq.expiration);
1278  prq.size = data_size;
1279  prq.type = type;
1280  process_reply (&prq,
1281  &query,
1282  pr);
1283 }
static char * expiration
Credential TTL.
Definition: gnunet-abd.c:96
static size_t data_size
Number of bytes in data.
Definition: gnunet-abd.c:187
struct GSF_CadetRequest * GSF_cadet_query(const struct GNUNET_PeerIdentity *target, const struct GNUNET_HashCode *query, enum GNUNET_BLOCK_Type type, GSF_CadetReplyProcessor proc, void *proc_cls)
Look for a block by directly contacting a particular peer.
static void cadet_reply_proc(void *cls, enum GNUNET_BLOCK_Type type, struct GNUNET_TIME_Absolute expiration, size_t data_size, const void *data)
Function called with a reply from the cadet.
enum GNUNET_GenericReturnValue GNUNET_BLOCK_check_block(struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, const void *block, size_t block_size)
Function called to validate a block.
Definition: block.c:299
enum GNUNET_GenericReturnValue GNUNET_BLOCK_get_key(struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, const void *block, size_t block_size, struct GNUNET_HashCode *key)
Function called to obtain the key for a block.
Definition: block.c:259
#define GNUNET_break_op(cond)
Use this for assertion violations caused by other peers (i.e.

References GSF_PendingRequest::cadet_request, GSF_PendingRequest::cadet_retry_count, CADET_RETRY_MAX, ProcessReplyClosure::data, data, data_size, expiration, ProcessReplyClosure::expiration, gettext_noop, GNUNET_BLOCK_check_block(), GNUNET_BLOCK_get_key(), GNUNET_BLOCK_TYPE_ANY, GNUNET_break, GNUNET_break_op, GNUNET_ERROR_TYPE_DEBUG, GNUNET_log, GNUNET_NO, GNUNET_OK, GNUNET_STATISTICS_update(), GNUNET_TIME_absolute_min(), GNUNET_TIME_relative_to_absolute(), GNUNET_TIME_UNIT_YEARS, GNUNET_YES, GSF_block_ctx, GSF_cadet_query(), GSF_stats, process_reply(), GSF_PendingRequest::public_data, GSF_PendingRequestData::query, ProcessReplyClosure::size, GSF_PendingRequestData::target, ProcessReplyClosure::type, GSF_PendingRequestData::type, and type.

Referenced by GSF_cadet_lookup_().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_cadet_lookup_()

void GSF_cadet_lookup_ ( struct GSF_PendingRequest pr)

Consider downloading via cadet (if possible)

Parameters
prthe pending request to process

Definition at line 1292 of file gnunet-service-fs_pr.c.

1293 {
1294  if (0 != pr->public_data.anonymity_level)
1295  return;
1296  if (0 == pr->public_data.target)
1297  {
1299  "Cannot do cadet-based download, target peer not known\n");
1300  return;
1301  }
1302  if (NULL != pr->cadet_request)
1303  return;
1305  &pr->public_data.query,
1306  pr->public_data.type,
1308  pr);
1309 }

References GSF_PendingRequestData::anonymity_level, cadet_reply_proc(), GSF_PendingRequest::cadet_request, GNUNET_ERROR_TYPE_DEBUG, GNUNET_log, GSF_cadet_query(), GSF_PendingRequest::public_data, GSF_PendingRequestData::query, GSF_PendingRequestData::target, and GSF_PendingRequestData::type.

Referenced by start_p2p_processing().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ warn_delay_task()

static void warn_delay_task ( void *  cls)
static

Task that issues a warning if the datastore lookup takes too long.

Parameters
clsthe struct GSF_PendingRequest

Definition at line 1318 of file gnunet-service-fs_pr.c.

1319 {
1320  struct GSF_PendingRequest *pr = cls;
1321 
1323  _ ("Datastore lookup already took %s!\n"),
1326  GNUNET_YES));
1328  &warn_delay_task,
1329  pr);
1330 }
static void warn_delay_task(void *cls)
Task that issues a warning if the datastore lookup takes too long.
@ GNUNET_ERROR_TYPE_WARNING
@ GNUNET_ERROR_TYPE_BULK
struct GNUNET_SCHEDULER_Task * GNUNET_SCHEDULER_add_delayed(struct GNUNET_TIME_Relative delay, GNUNET_SCHEDULER_TaskCallback task, void *task_cls)
Schedule a new task to be run with a specified delay.
Definition: scheduler.c:1254
#define _(String)
GNU gettext support macro.
Definition: platform.h:177
struct GNUNET_TIME_Absolute qe_start
Time we started the last datastore lookup.

References _, GNUNET_ERROR_TYPE_BULK, GNUNET_ERROR_TYPE_WARNING, GNUNET_log, GNUNET_SCHEDULER_add_delayed(), GNUNET_STRINGS_relative_time_to_string(), GNUNET_TIME_absolute_get_duration(), GNUNET_TIME_UNIT_MINUTES, GNUNET_YES, GSF_PendingRequest::qe_start, and GSF_PendingRequest::warn_task.

Referenced by start_local_query().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ odc_warn_delay_task()

static void odc_warn_delay_task ( void *  cls)
static

Task that issues a warning if the datastore lookup takes too long.

Parameters
clsthe struct GSF_PendingRequest

Definition at line 1339 of file gnunet-service-fs_pr.c.

1340 {
1341  struct GSF_PendingRequest *pr = cls;
1342 
1344  _ ("On-demand lookup already took %s!\n"),
1347  GNUNET_YES));
1350  pr);
1351 }
static void odc_warn_delay_task(void *cls)
Task that issues a warning if the datastore lookup takes too long.

References _, GNUNET_ERROR_TYPE_WARNING, GNUNET_log, GNUNET_SCHEDULER_add_delayed(), GNUNET_STRINGS_relative_time_to_string(), GNUNET_TIME_absolute_get_duration(), GNUNET_TIME_UNIT_MINUTES, GNUNET_YES, GSF_PendingRequest::qe_start, and GSF_PendingRequest::warn_task.

Referenced by process_local_reply().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ call_continuation()

static void call_continuation ( struct GSF_PendingRequest pr)
static

Definition at line 1356 of file gnunet-service-fs_pr.c.

1357 {
1359 
1360  GNUNET_assert (NULL == pr->qe);
1361  if (NULL != pr->warn_task)
1362  {
1364  pr->warn_task = NULL;
1365  }
1366  if (NULL == cont)
1367  return; /* no continuation */
1368  pr->llc_cont = NULL;
1369  if (0 != (GSF_PRO_LOCAL_ONLY & pr->public_data.options))
1370  {
1372  {
1373  /* Signal that we are done and that there won't be any
1374  additional results to allow client to clean up state. */
1375  pr->rh (pr->rh_cls,
1377  pr,
1378  UINT32_MAX,
1382  NULL,
1383  0);
1384  }
1385  /* Finally, call our continuation to signal that we are
1386  done with local processing of this request; i.e. to
1387  start reading again from the client. */
1388  cont (pr->llc_cont_cls,
1389  NULL,
1391  return;
1392  }
1393 
1394  cont (pr->llc_cont_cls,
1395  pr,
1396  pr->local_result);
1397 }
@ GSF_PRO_LOCAL_ONLY
Request must only be processed locally.

References GNUNET_assert, GNUNET_BLOCK_REPLY_OK_LAST, GNUNET_BLOCK_TYPE_ANY, GNUNET_SCHEDULER_cancel(), GNUNET_TIME_UNIT_FOREVER_ABS, GNUNET_TIME_UNIT_ZERO_ABS, GSF_PRO_LOCAL_ONLY, GSF_PendingRequest::llc_cont, GSF_PendingRequest::llc_cont_cls, GSF_PendingRequest::local_result, GSF_PendingRequestData::options, GSF_PendingRequest::public_data, GSF_PendingRequest::qe, GSF_PendingRequest::rh, GSF_PendingRequest::rh_cls, and GSF_PendingRequest::warn_task.

Referenced by no_more_local_results(), process_local_reply(), and start_local_query().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ no_more_local_results()

static void no_more_local_results ( struct GSF_PendingRequest pr)
static

Definition at line 1402 of file gnunet-service-fs_pr.c.

1403 {
1405  "No further local responses available.\n");
1406 #if INSANE_STATISTICS
1410  gettext_noop (
1411  "# requested DBLOCK or IBLOCK not found"),
1412  1,
1413  GNUNET_NO);
1414 #endif
1415  call_continuation (pr);
1416 }
static void call_continuation(struct GSF_PendingRequest *pr)
@ GNUNET_BLOCK_TYPE_FS_DBLOCK
Data block (leaf) in the CHK tree.
@ GNUNET_BLOCK_TYPE_FS_IBLOCK
Inner block in the CHK tree.

References call_continuation(), gettext_noop, GNUNET_BLOCK_TYPE_FS_DBLOCK, GNUNET_BLOCK_TYPE_FS_IBLOCK, GNUNET_ERROR_TYPE_BULK, GNUNET_ERROR_TYPE_DEBUG, GNUNET_log, GNUNET_NO, GNUNET_STATISTICS_update(), GSF_stats, GSF_PendingRequest::public_data, and GSF_PendingRequestData::type.

Referenced by process_local_reply().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ process_local_reply()

static void process_local_reply ( void *  cls,
const struct GNUNET_HashCode key,
size_t  size,
const void *  data,
enum GNUNET_BLOCK_Type  type,
uint32_t  priority,
uint32_t  anonymity,
uint32_t  replication,
struct GNUNET_TIME_Absolute  expiration,
uint64_t  uid 
)
static

We're processing (local) results for a search request from another peer.

Pass applicable results to the peer and if we are done either clean up (operation complete) or forward to other peers (more results possible).

Parameters
clsour closure (struct GSF_PendingRequest *)
keykey for the content
sizenumber of bytes in data
datacontent stored
typetype of the content
prioritypriority of the content
anonymityanonymity-level for the content
replicationreplication-level for the content
expirationexpiration time for the content
uidunique identifier for the datum; maybe 0 if no unique identifier is available

Definition at line 1499 of file gnunet-service-fs_pr.c.

1509 {
1510  struct GSF_PendingRequest *pr = cls;
1511  struct ProcessReplyClosure prq;
1512  struct GNUNET_HashCode query;
1513  unsigned int old_rf;
1514 
1516  pr->warn_task = NULL;
1517  if (NULL == pr->qe)
1518  goto called_from_on_demand;
1519  pr->qe = NULL;
1520  if (
1521  (NULL == key) && pr->seen_null &&
1522  ! pr->have_first_uid) /* We have hit the end for the 2nd time with no results */
1523  {
1524  /* No results */
1525 #if INSANE_STATISTICS
1527  gettext_noop (
1528  "# Datastore lookups concluded (no results)"),
1529  1,
1530  GNUNET_NO);
1531 #endif
1532  no_more_local_results (pr);
1533  return;
1534  }
1535  if (((NULL == key) &&
1536  pr->seen_null) || /* We have hit the end for the 2nd time OR */
1537  (pr->seen_null && pr->have_first_uid &&
1538  (uid >= pr->first_uid))) /* We have hit the end and past first UID */
1539  {
1540  /* Seen all results */
1542  gettext_noop (
1543  "# Datastore lookups concluded (seen all)"),
1544  1,
1545  GNUNET_NO);
1546  no_more_local_results (pr);
1547  return;
1548  }
1549  if (NULL == key)
1550  {
1551  GNUNET_assert (! pr->seen_null);
1552  pr->seen_null = true;
1553  start_local_query (pr, 0 /* next_uid */, false /* random */);
1554  return;
1555  }
1556  if (! pr->have_first_uid)
1557  {
1558  pr->first_uid = uid;
1559  pr->have_first_uid = true;
1560  }
1561  pr->result_count++;
1562  if (pr->result_count > MAX_RESULTS)
1563  {
1565  GSF_stats,
1566  gettext_noop ("# Datastore lookups aborted (more than MAX_RESULTS)"),
1567  1,
1568  GNUNET_NO);
1569  no_more_local_results (pr);
1570  return;
1571  }
1573  "Received reply for `%s' of type %d with UID %llu from datastore.\n",
1574  GNUNET_h2s (key),
1575  type,
1576  (unsigned long long) uid);
1578  {
1580  "Found ONDEMAND block, performing on-demand encoding\n");
1582  gettext_noop (
1583  "# on-demand blocks matched requests"),
1584  1,
1585  GNUNET_NO);
1589  pr);
1591  size,
1592  data,
1593  type,
1594  priority,
1595  anonymity,
1596  replication,
1597  expiration,
1598  uid,
1600  pr))
1601  {
1603  gettext_noop (
1604  "# on-demand lookups performed successfully"),
1605  1,
1606  GNUNET_NO);
1607  return; /* we're done */
1608  }
1610  gettext_noop ("# on-demand lookups failed"),
1611  1,
1612  GNUNET_NO);
1614  start_local_query (pr, uid + 1 /* next_uid */, false /* random */);
1615  return;
1616  }
1617 called_from_on_demand:
1618  old_rf = pr->public_data.results_found;
1619  memset (&prq, 0, sizeof(prq));
1620  prq.data = data;
1621  prq.expiration = expiration;
1622  prq.size = size;
1623  if (GNUNET_OK !=
1625  type,
1626  data,
1627  size,
1628  &query))
1629  {
1630  GNUNET_break (0);
1632  key,
1633  size,
1634  data,
1635  UINT_MAX,
1636  UINT_MAX,
1637  NULL,
1638  NULL);
1639  start_local_query (pr, uid + 1 /* next_uid */, false /* random */);
1640  return;
1641  }
1642  prq.type = type;
1643  prq.priority = priority;
1644  prq.request_found = GNUNET_NO;
1645  prq.anonymity_level = anonymity;
1646  if ((0 == old_rf) && (0 == pr->public_data.results_found))
1648  process_reply (&prq,
1649  key,
1650  pr);
1651  pr->local_result = prq.eval;
1652  if (GNUNET_BLOCK_REPLY_OK_LAST == prq.eval)
1653  {
1655  GSF_stats,
1656  gettext_noop ("# Datastore lookups concluded (found last result)"),
1657  1,
1658  GNUNET_NO);
1659  call_continuation (pr);
1660  return;
1661  }
1662  if ((0 == (GSF_PRO_PRIORITY_UNLIMITED & pr->public_data.options)) &&
1664  (pr->public_data.results_found > 5 + 2 * pr->public_data.priority)))
1665  {
1666  GNUNET_log (GNUNET_ERROR_TYPE_DEBUG, "Load too high, done with request\n");
1668  gettext_noop (
1669  "# Datastore lookups concluded (load too high)"),
1670  1,
1671  GNUNET_NO);
1672  call_continuation (pr);
1673  return;
1674  }
1675  start_local_query (pr, uid + 1 /* next_uid */, false /* random */);
1676 }
static unsigned int replication
static unsigned int anonymity
void GSF_update_datastore_delay_(struct GNUNET_TIME_Absolute start)
We've just now completed a datastore request.
int GSF_test_get_load_too_high_(uint32_t priority)
Test if the DATABASE (GET) load on this peer is too high to even consider processing the query at all...
int GNUNET_FS_handle_on_demand_block(const struct GNUNET_HashCode *key, uint32_t size, const void *data, enum GNUNET_BLOCK_Type type, uint32_t priority, uint32_t anonymity, uint32_t replication, struct GNUNET_TIME_Absolute expiration, uint64_t uid, GNUNET_DATASTORE_DatumProcessor cont, void *cont_cls)
We've received an on-demand encoded block from the datastore.
#define MAX_RESULTS
Hard limit on the number of results we may get from the datastore per query.
static void no_more_local_results(struct GSF_PendingRequest *pr)
static void process_local_reply(void *cls, const struct GNUNET_HashCode *key, size_t size, const void *data, enum GNUNET_BLOCK_Type type, uint32_t priority, uint32_t anonymity, uint32_t replication, struct GNUNET_TIME_Absolute expiration, uint64_t uid)
We're processing (local) results for a search request from another peer.
static void start_local_query(struct GSF_PendingRequest *pr, uint64_t next_uid, bool random)
@ GSF_PRO_PRIORITY_UNLIMITED
Request priority is allowed to be exceeded.
@ GNUNET_BLOCK_TYPE_FS_ONDEMAND
Type of a block representing a block to be encoded on demand from disk.
struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_remove(struct GNUNET_DATASTORE_Handle *h, const struct GNUNET_HashCode *key, size_t size, const void *data, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_ContinuationWithStatus cont, void *cont_cls)
Explicitly remove some content from the database.
size_t result_count
Result count.
uint64_t first_uid
Unique ID of the first result from the local datastore; used to terminate the loop.
bool seen_null
Have we seen a NULL result yet?
bool have_first_uid
Do we have a first UID yet?

References anonymity, ProcessReplyClosure::anonymity_level, call_continuation(), ProcessReplyClosure::data, data, ProcessReplyClosure::eval, expiration, ProcessReplyClosure::expiration, GSF_PendingRequest::first_uid, gettext_noop, GNUNET_assert, GNUNET_BLOCK_get_key(), GNUNET_BLOCK_REPLY_OK_LAST, GNUNET_BLOCK_TYPE_FS_ONDEMAND, GNUNET_break, GNUNET_DATASTORE_remove(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_FS_handle_on_demand_block(), GNUNET_h2s(), GNUNET_log, GNUNET_NO, GNUNET_OK, GNUNET_SCHEDULER_add_delayed(), GNUNET_SCHEDULER_cancel(), GNUNET_STATISTICS_update(), GNUNET_TIME_absolute_get(), GNUNET_TIME_UNIT_MINUTES, GNUNET_YES, GSF_block_ctx, GSF_dsh, GSF_PRO_PRIORITY_UNLIMITED, GSF_stats, GSF_test_get_load_too_high_(), GSF_update_datastore_delay_(), GSF_PendingRequest::have_first_uid, key, GSF_PendingRequest::local_result, MAX_RESULTS, no_more_local_results(), odc_warn_delay_task(), GSF_PendingRequestData::options, ProcessReplyClosure::priority, GSF_PendingRequestData::priority, process_reply(), GSF_PendingRequest::public_data, GSF_PendingRequest::qe, GSF_PendingRequest::qe_start, replication, ProcessReplyClosure::request_found, GSF_PendingRequest::result_count, GSF_PendingRequestData::results_found, GSF_PendingRequest::seen_null, ProcessReplyClosure::size, size, start_local_query(), GSF_PendingRequestData::start_time, ProcessReplyClosure::type, type, and GSF_PendingRequest::warn_task.

Referenced by start_local_query().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ start_local_query()

static void start_local_query ( struct GSF_PendingRequest pr,
uint64_t  next_uid,
bool  random 
)
static

Definition at line 1435 of file gnunet-service-fs_pr.c.

1438 {
1441  &warn_delay_task,
1442  pr);
1444  next_uid,
1445  random,
1446  &pr->public_data.query,
1447  pr->public_data.type ==
1450  : pr->public_data.type,
1452  & pr->public_data.options))
1453  ? UINT_MAX
1454  : 1
1455  /* queue priority */,
1457  & pr->public_data.options))
1458  ? UINT_MAX
1460  /* max queue size */,
1462  pr);
1463  if (NULL != pr->qe)
1464  return;
1465  GNUNET_log (
1467  "ERROR Requesting `%s' of type %d with next_uid %llu from datastore.\n",
1468  GNUNET_h2s (&pr->public_data.query),
1469  pr->public_data.type,
1470  (unsigned long long) next_uid);
1472  gettext_noop (
1473  "# Datastore lookups concluded (error queueing)"),
1474  1,
1475  GNUNET_NO);
1476  call_continuation (pr);
1477 }
unsigned int GSF_datastore_queue_size
Size of the datastore queue we assume for common requests.
struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_get_key(struct GNUNET_DATASTORE_Handle *h, uint64_t next_uid, bool random, const struct GNUNET_HashCode *key, enum GNUNET_BLOCK_Type type, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_DatumProcessor proc, void *proc_cls)
Get a result for a particular key from the datastore.

References call_continuation(), gettext_noop, GNUNET_BLOCK_TYPE_ANY, GNUNET_BLOCK_TYPE_FS_DBLOCK, GNUNET_DATASTORE_get_key(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_h2s(), GNUNET_log, GNUNET_NO, GNUNET_SCHEDULER_add_delayed(), GNUNET_STATISTICS_update(), GNUNET_TIME_absolute_get(), GNUNET_TIME_UNIT_MINUTES, GSF_datastore_queue_size, GSF_dsh, GSF_PRO_PRIORITY_UNLIMITED, GSF_stats, GSF_PendingRequestData::options, process_local_reply(), GSF_PendingRequest::public_data, GSF_PendingRequest::qe, GSF_PendingRequest::qe_start, GSF_PendingRequestData::query, GSF_PendingRequestData::type, warn_delay_task(), and GSF_PendingRequest::warn_task.

Referenced by GSF_local_lookup_(), and process_local_reply().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_pending_request_test_target_()

int GSF_pending_request_test_target_ ( struct GSF_PendingRequest pr,
const struct GNUNET_PeerIdentity target 
)

Is the given target a legitimate peer for forwarding the given request?

Parameters
prrequest
target
Returns
GNUNET_YES if this request could be forwarded to the given peer

Definition at line 1687 of file gnunet-service-fs_pr.c.

1689 {
1690  struct GNUNET_PeerIdentity pi;
1691 
1692  if (0 == pr->origin_pid)
1693  return GNUNET_YES;
1695  return (0 == memcmp (&pi, target, sizeof(struct GNUNET_PeerIdentity)))
1696  ? GNUNET_NO
1697  : GNUNET_YES;
1698 }

References GNUNET_NO, GNUNET_PEER_resolve(), GNUNET_YES, GSF_PendingRequest::origin_pid, and pi.

Referenced by consider_peer_for_forwarding(), and consider_request_for_forwarding().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_local_lookup_()

void GSF_local_lookup_ ( struct GSF_PendingRequest pr,
GSF_LocalLookupContinuation  cont,
void *  cont_cls 
)

Look up the request in the local datastore.

Parameters
prthe pending request to process
contfunction to call at the end
cont_clsclosure for cont

Definition at line 1709 of file gnunet-service-fs_pr.c.

1712 {
1713  GNUNET_assert (NULL == pr->gh);
1714  GNUNET_assert (NULL == pr->cadet_request);
1715  GNUNET_assert (NULL == pr->llc_cont);
1716  pr->llc_cont = cont;
1717  pr->llc_cont_cls = cont_cls;
1718 #if INSANE_STATISTICS
1720  gettext_noop ("# Datastore lookups initiated"),
1721  1,
1722  GNUNET_NO);
1723 #endif
1724  start_local_query (pr, 0 /* next_uid */, true /* random */);
1725 }

References GSF_PendingRequest::cadet_request, gettext_noop, GSF_PendingRequest::gh, GNUNET_assert, GNUNET_NO, GNUNET_STATISTICS_update(), GSF_stats, GSF_PendingRequest::llc_cont, GSF_PendingRequest::llc_cont_cls, and start_local_query().

Referenced by handle_client_start_search(), and handle_p2p_get().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ handle_p2p_put()

void handle_p2p_put ( void *  cls,
const struct PutMessage put 
)

Handle P2P "CONTENT" message.

Checks that the message is well-formed and then checks if there are any pending requests for this content and possibly passes it on (to local clients or other peers). Does NOT perform migration (content caching at this peer).

Parameters
clsthe other peer involved
putthe actual message

Definition at line 1738 of file gnunet-service-fs_pr.c.

1740 {
1741  struct GSF_ConnectedPeer *cp = cls;
1742  uint16_t msize;
1743  size_t dsize;
1744  enum GNUNET_BLOCK_Type type;
1746  struct GNUNET_HashCode query;
1747  struct ProcessReplyClosure prq;
1748  struct GNUNET_TIME_Relative block_time;
1749  double putl;
1750  struct PutMigrationContext *pmc;
1751 
1753  "Received P2P PUT from %s\n",
1756  msize = ntohs (put->header.size);
1757  dsize = msize - sizeof(struct PutMessage);
1758  type = ntohl (put->type);
1760  /* do not allow migrated content to live longer than 1 year */
1763  expiration);
1764  if (GNUNET_OK !=
1766  type,
1767  &put[1],
1768  dsize))
1769  {
1770  GNUNET_break_op (0);
1771  return;
1772  }
1773  if (GNUNET_OK !=
1775  type,
1776  &put[1],
1777  dsize,
1778  &query))
1779  {
1780  GNUNET_break_op (0);
1781  return;
1782  }
1784  gettext_noop ("# GAP PUT messages received"),
1785  1,
1786  GNUNET_NO);
1787  /* now, lookup 'query' */
1788  prq.data = (const void *) &put[1];
1789  prq.sender = cp;
1790  prq.size = dsize;
1791  prq.type = type;
1792  prq.expiration = expiration;
1793  prq.priority = 0;
1794  prq.anonymity_level = UINT32_MAX;
1795  prq.request_found = GNUNET_NO;
1797  &query,
1798  &process_reply,
1799  &prq);
1800  if (NULL != cp)
1801  {
1804  + 1000 * prq.priority);
1805  GSF_get_peer_performance_data_ (cp)->respect += prq.priority;
1806  }
1807  if ((GNUNET_YES == active_to_migration) && (NULL != cp) &&
1808  (GNUNET_NO == test_put_load_too_high (prq.priority)))
1809  {
1811  "Replicating result for query `%s' with priority %u\n",
1812  GNUNET_h2s (&query),
1813  prq.priority);
1814  pmc = GNUNET_new (struct PutMigrationContext);
1815  pmc->start = GNUNET_TIME_absolute_get ();
1816  pmc->requested = prq.request_found;
1819  &pmc->origin);
1820  if (NULL == GNUNET_DATASTORE_put (GSF_dsh,
1821  0,
1822  &query,
1823  dsize,
1824  &put[1],
1825  type,
1826  prq.priority,
1827  1 /* anonymity */,
1828  0 /* replication */,
1829  expiration,
1830  1 + prq.priority,
1833  pmc))
1834  {
1836  GNUNET_SYSERR,
1838  NULL);
1839  }
1840  }
1841  else if (NULL != cp)
1842  {
1844  "Choosing not to keep content `%s' (%d/%d)\n",
1845  GNUNET_h2s (&query),
1847  test_put_load_too_high (prq.priority));
1848  }
1850  if ((NULL != cp) && (GNUNET_NO == prq.request_found) &&
1852  (putl > 2.5 * (1 + prq.priority))))
1853  {
1856  block_time = GNUNET_TIME_relative_multiply (
1859  (unsigned int) (60000 * putl * putl)));
1860  GNUNET_log (
1862  "Asking to stop migration for %s because of load %f and events %d/%d\n",
1864  putl,
1866  (GNUNET_NO == prq.request_found));
1868  GNUNET_TIME_relative_to_absolute (block_time));
1869  }
1870 }
unsigned int GSF_cover_content_count
How many content messages have we received 'recently' that have not yet been claimed as cover traffic...
void GSF_connected_peer_change_preference_(struct GSF_ConnectedPeer *cp, uint64_t pref)
Notify core about a preference we have for the given peer (to allocate more resources towards it).
#define CONTENT_BANDWIDTH_VALUE
Bandwidth value of a 0-priority content (must be fairly high compared to query since content is typic...
static struct GNUNET_PeerIdentity pid
Identity of the peer we transmit to / connect to.
GNUNET_BLOCK_Type
WARNING: This header is generated! In order to add DHT block types, you must register them in GANA,...
int GNUNET_CONTAINER_multihashmap_get_multiple(struct GNUNET_CONTAINER_MultiHashMap *map, const struct GNUNET_HashCode *key, GNUNET_CONTAINER_MulitHashMapIteratorCallback it, void *it_cls)
Iterate over all entries in the map that match a particular key.
const char * GNUNET_i2s(const struct GNUNET_PeerIdentity *pid)
Convert a peer identity to a string (for printing debug messages).
#define GNUNET_TIME_UNIT_MILLISECONDS
One millisecond.
struct GNUNET_TIME_Absolute GNUNET_TIME_absolute_ntoh(struct GNUNET_TIME_AbsoluteNBO a)
Convert absolute time from network byte order.
Definition: time.c:736
uint16_t size
The length of the struct (in bytes, including the length field itself), in big-endian format.
uint32_t respect
Respect rating for this peer.
Response from FS service with a result for a previous FS search.
Definition: fs.h:329
struct GNUNET_MessageHeader header
Message type will be GNUNET_MESSAGE_TYPE_FS_PUT.
Definition: fs.h:333
uint32_t type
Type of the block (in big endian).
Definition: fs.h:338
struct GNUNET_TIME_AbsoluteNBO expiration
When does this result expire?
Definition: fs.h:343
struct GNUNET_TESTBED_Peer * peer
The peer associated with this model.

References active_to_migration, ProcessReplyClosure::anonymity_level, CONTENT_BANDWIDTH_VALUE, ProcessReplyClosure::data, datastore_put_load, expiration, PutMessage::expiration, ProcessReplyClosure::expiration, gettext_noop, GNUNET_assert, GNUNET_BLOCK_check_block(), GNUNET_BLOCK_get_key(), GNUNET_break_op, GNUNET_CONTAINER_multihashmap_get_multiple(), GNUNET_CRYPTO_QUALITY_WEAK, GNUNET_CRYPTO_random_u32(), GNUNET_DATASTORE_put(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_h2s(), GNUNET_i2s(), GNUNET_LOAD_get_load(), GNUNET_log, GNUNET_new, GNUNET_NO, GNUNET_OK, GNUNET_PEER_resolve(), GNUNET_STATISTICS_update(), GNUNET_STRINGS_relative_time_to_string(), GNUNET_SYSERR, GNUNET_TIME_absolute_get(), GNUNET_TIME_absolute_min(), GNUNET_TIME_absolute_ntoh(), GNUNET_TIME_relative_multiply(), GNUNET_TIME_relative_to_absolute(), GNUNET_TIME_UNIT_MILLISECONDS, GNUNET_TIME_UNIT_YEARS, GNUNET_TIME_UNIT_ZERO_ABS, GNUNET_YES, GSF_block_ctx, GSF_block_peer_migration_(), GSF_connected_peer_change_preference_(), GSF_cover_content_count, GSF_dsh, GSF_get_peer_performance_data_(), GSF_stats, PutMessage::header, MAX_DATASTORE_QUEUE, PutMigrationContext::origin, peer, pid, pr_map, ProcessReplyClosure::priority, process_reply(), put_migration_continuation(), ProcessReplyClosure::request_found, PutMigrationContext::requested, GSF_PeerPerformanceData::respect, ProcessReplyClosure::sender, ProcessReplyClosure::size, GNUNET_MessageHeader::size, PutMigrationContext::start, test_put_load_too_high(), PutMessage::type, ProcessReplyClosure::type, and type.

Here is the call graph for this function:

◆ GSF_pending_request_test_active_()

int GSF_pending_request_test_active_ ( struct GSF_PendingRequest pr)

Check if the given request is still active.

Parameters
prpending request
Returns
GNUNET_YES if the request is still active

Definition at line 1880 of file gnunet-service-fs_pr.c.

1881 {
1882  return (NULL != pr->rh) ? GNUNET_YES : GNUNET_NO;
1883 }

References GNUNET_NO, GNUNET_YES, and GSF_PendingRequest::rh.

Referenced by consider_peer_for_forwarding(), get_latest(), GSF_consider_forwarding(), GSF_plan_add_(), and merge_pr().

Here is the caller graph for this function:

◆ GSF_pending_request_init_()

void GSF_pending_request_init_ ( void  )

Setup the subsystem.

Definition at line 1890 of file gnunet-service-fs_pr.c.

1891 {
1892  if (GNUNET_OK !=
1894  "fs",
1895  "MAX_PENDING_REQUESTS",
1897  {
1899  "fs",
1900  "MAX_PENDING_REQUESTS");
1901  }
1903  GNUNET_CONFIGURATION_get_value_yesno (GSF_cfg, "FS", "CONTENT_CACHING");
1908 }
const struct GNUNET_CONFIGURATION_Handle * GSF_cfg
Our configuration.
#define DATASTORE_LOAD_AUTODECLINE
At what frequency should our datastore load decrease automatically (since if we don't use it,...
enum GNUNET_GenericReturnValue GNUNET_CONFIGURATION_get_value_number(const struct GNUNET_CONFIGURATION_Handle *cfg, const char *section, const char *option, unsigned long long *number)
Get a configuration value that should be a number.
enum GNUNET_GenericReturnValue GNUNET_CONFIGURATION_get_value_yesno(const struct GNUNET_CONFIGURATION_Handle *cfg, const char *section, const char *option)
Get a configuration value that should be in a set of "YES" or "NO".
struct GNUNET_CONTAINER_MultiHashMap * GNUNET_CONTAINER_multihashmap_create(unsigned int len, int do_not_copy_keys)
Create a multi hash map.
struct GNUNET_CONTAINER_Heap * GNUNET_CONTAINER_heap_create(enum GNUNET_CONTAINER_HeapOrder order)
Create a new heap.
@ GNUNET_CONTAINER_HEAP_ORDER_MIN
Heap with the minimum cost at the root.
struct GNUNET_LOAD_Value * GNUNET_LOAD_value_init(struct GNUNET_TIME_Relative autodecline)
Create a new load value.
Definition: load.c:124
void GNUNET_log_config_missing(enum GNUNET_ErrorType kind, const char *section, const char *option)
Log error message about missing configuration option.
@ GNUNET_ERROR_TYPE_INFO

References active_to_migration, DATASTORE_LOAD_AUTODECLINE, datastore_put_load, GNUNET_CONFIGURATION_get_value_number(), GNUNET_CONFIGURATION_get_value_yesno(), GNUNET_CONTAINER_heap_create(), GNUNET_CONTAINER_HEAP_ORDER_MIN, GNUNET_CONTAINER_multihashmap_create(), GNUNET_ERROR_TYPE_INFO, GNUNET_LOAD_value_init(), GNUNET_log_config_missing(), GNUNET_OK, GNUNET_YES, GSF_cfg, max_pending_requests, pr_map, and requests_by_expiration_heap.

Referenced by run().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GSF_pending_request_done_()

void GSF_pending_request_done_ ( void  )

Shutdown the subsystem.

Definition at line 1915 of file gnunet-service-fs_pr.c.

1916 {
1919  pr_map = NULL;
1923  datastore_put_load = NULL;
1924 }
void GNUNET_CONTAINER_multihashmap_destroy(struct GNUNET_CONTAINER_MultiHashMap *map)
Destroy a hash map.
void GNUNET_CONTAINER_heap_destroy(struct GNUNET_CONTAINER_Heap *heap)
Destroys the heap.
#define GNUNET_LOAD_value_free(lv)
Free a load value.

Referenced by shutdown_task().

Here is the caller graph for this function:

Variable Documentation

◆ pr_map

struct GNUNET_CONTAINER_MultiHashMap* pr_map
static

All pending requests, ordered by the query.

Entries are of type 'struct GSF_PendingRequest*'.

Definition at line 205 of file gnunet-service-fs_pr.c.

Referenced by clean_request(), GSF_iterate_pending_requests_(), GSF_pending_request_cancel_(), GSF_pending_request_create_(), GSF_pending_request_init_(), and handle_p2p_put().

◆ datastore_put_load

struct GNUNET_LOAD_Value* datastore_put_load
static

Datastore 'PUT' load tracking.

Definition at line 211 of file gnunet-service-fs_pr.c.

Referenced by GSF_pending_request_init_(), handle_p2p_put(), put_migration_continuation(), and test_put_load_too_high().

◆ active_to_migration

int active_to_migration
static

Are we allowed to migrate content to this peer.

Definition at line 217 of file gnunet-service-fs_pr.c.

Referenced by GSF_pending_request_init_(), handle_dht_reply(), and handle_p2p_put().

◆ requests_by_expiration_heap

struct GNUNET_CONTAINER_Heap* requests_by_expiration_heap
static

Heap with the request that will expire next at the top.

Contains pointers of type "struct PendingRequest*"; these will also be aliased from the "requests_by_peer" data structures and the "requests_by_query" table. Note that requests from our clients don't expire and are thus NOT in the "requests_by_expiration" (or the "requests_by_peer" tables).

Definition at line 228 of file gnunet-service-fs_pr.c.

Referenced by GSF_pending_request_create_(), and GSF_pending_request_init_().

◆ max_pending_requests

unsigned long long max_pending_requests = (32 * 1024)
static

Maximum number of requests (from other peers, overall) that we're willing to have pending at any given point in time.

Can be changed via the configuration file (32k is just the default).

Definition at line 236 of file gnunet-service-fs_pr.c.

Referenced by GSF_pending_request_create_(), and GSF_pending_request_init_().