GNUnet 0.21.0

Data store for files stored on a GNUnet node. More...

Collaboration diagram for Data Store service:

Macros

#define GNUNET_DATASTORE_MAX_VALUE_SIZE   65536
 Maximum size of a value that can be stored in the datastore. More...
 

Typedefs

typedef void(* GNUNET_DATASTORE_ContinuationWithStatus) (void *cls, int32_t success, struct GNUNET_TIME_Absolute min_expiration, const char *msg)
 Continuation called to notify client about result of the operation. More...
 
typedef void(* GNUNET_DATASTORE_DatumProcessor) (void *cls, const struct GNUNET_HashCode *key, size_t size, const void *data, enum GNUNET_BLOCK_Type type, uint32_t priority, uint32_t anonymity, uint32_t replication, struct GNUNET_TIME_Absolute expiration, uint64_t uid)
 Process a datum that was stored in the datastore. More...
 

Functions

struct GNUNET_DATASTORE_HandleGNUNET_DATASTORE_connect (const struct GNUNET_CONFIGURATION_Handle *cfg)
 Connect to the datastore service. More...
 
void GNUNET_DATASTORE_disconnect (struct GNUNET_DATASTORE_Handle *h, int drop)
 Disconnect from the datastore service (and free associated resources). More...
 
struct GNUNET_DATASTORE_QueueEntryGNUNET_DATASTORE_reserve (struct GNUNET_DATASTORE_Handle *h, uint64_t amount, uint32_t entries, GNUNET_DATASTORE_ContinuationWithStatus cont, void *cont_cls)
 Reserve space in the datastore. More...
 
struct GNUNET_DATASTORE_QueueEntryGNUNET_DATASTORE_put (struct GNUNET_DATASTORE_Handle *h, uint32_t rid, const struct GNUNET_HashCode *key, size_t size, const void *data, enum GNUNET_BLOCK_Type type, uint32_t priority, uint32_t anonymity, uint32_t replication, struct GNUNET_TIME_Absolute expiration, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_ContinuationWithStatus cont, void *cont_cls)
 Store an item in the datastore. More...
 
struct GNUNET_DATASTORE_QueueEntryGNUNET_DATASTORE_release_reserve (struct GNUNET_DATASTORE_Handle *h, uint32_t rid, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_ContinuationWithStatus cont, void *cont_cls)
 Signal that all of the data for which a reservation was made has been stored and that whatever excess space might have been reserved can now be released. More...
 
struct GNUNET_DATASTORE_QueueEntryGNUNET_DATASTORE_remove (struct GNUNET_DATASTORE_Handle *h, const struct GNUNET_HashCode *key, size_t size, const void *data, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_ContinuationWithStatus cont, void *cont_cls)
 Explicitly remove some content from the database. More...
 
struct GNUNET_DATASTORE_QueueEntryGNUNET_DATASTORE_get_key (struct GNUNET_DATASTORE_Handle *h, uint64_t next_uid, bool random, const struct GNUNET_HashCode *key, enum GNUNET_BLOCK_Type type, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_DatumProcessor proc, void *proc_cls)
 Get a result for a particular key from the datastore. More...
 
struct GNUNET_DATASTORE_QueueEntryGNUNET_DATASTORE_get_zero_anonymity (struct GNUNET_DATASTORE_Handle *h, uint64_t next_uid, unsigned int queue_priority, unsigned int max_queue_size, enum GNUNET_BLOCK_Type type, GNUNET_DATASTORE_DatumProcessor proc, void *proc_cls)
 Get a single zero-anonymity value from the datastore. More...
 
struct GNUNET_DATASTORE_QueueEntryGNUNET_DATASTORE_get_for_replication (struct GNUNET_DATASTORE_Handle *h, unsigned int queue_priority, unsigned int max_queue_size, GNUNET_DATASTORE_DatumProcessor proc, void *proc_cls)
 Get a random value from the datastore for content replication. More...
 
void GNUNET_DATASTORE_cancel (struct GNUNET_DATASTORE_QueueEntry *qe)
 Cancel a datastore operation. More...
 

Detailed Description

Data store for files stored on a GNUnet node.

Provides an API that can be used manage the datastore for files stored on a GNUnet node. Note that the datastore is NOT responsible for on-demand encoding, that is achieved using a special kind of entry.

Macro Definition Documentation

◆ GNUNET_DATASTORE_MAX_VALUE_SIZE

#define GNUNET_DATASTORE_MAX_VALUE_SIZE   65536

Maximum size of a value that can be stored in the datastore.

Definition at line 70 of file gnunet_datastore_service.h.

Typedef Documentation

◆ GNUNET_DATASTORE_ContinuationWithStatus

typedef void(* GNUNET_DATASTORE_ContinuationWithStatus) (void *cls, int32_t success, struct GNUNET_TIME_Absolute min_expiration, const char *msg)

Continuation called to notify client about result of the operation.

Parameters
clsclosure
successGNUNET_SYSERR on failure GNUNET_NO if content was already there GNUNET_YES (or other positive value) on success
min_expirationminimum expiration time required for 0-priority content to be stored by the datacache at this time, zero for unknown, forever if we have no space for 0-priority content
msgNULL on success, otherwise an error message

Definition at line 107 of file gnunet_datastore_service.h.

◆ GNUNET_DATASTORE_DatumProcessor

typedef void(* GNUNET_DATASTORE_DatumProcessor) (void *cls, const struct GNUNET_HashCode *key, size_t size, const void *data, enum GNUNET_BLOCK_Type type, uint32_t priority, uint32_t anonymity, uint32_t replication, struct GNUNET_TIME_Absolute expiration, uint64_t uid)

Process a datum that was stored in the datastore.

Parameters
clsclosure
keykey for the content
sizenumber of bytes in data
datacontent stored
typetype of the content
prioritypriority of the content
anonymityanonymity-level for the content
replicationhow often should the content be replicated to other peers?
expirationexpiration time for the content
uidunique identifier for the datum; maybe 0 if no unique identifier is available

Definition at line 253 of file gnunet_datastore_service.h.

Function Documentation

◆ GNUNET_DATASTORE_connect()

struct GNUNET_DATASTORE_Handle * GNUNET_DATASTORE_connect ( const struct GNUNET_CONFIGURATION_Handle cfg)

Connect to the datastore service.

Parameters
cfgconfiguration to use
Returns
handle to use to access the service

Definition at line 371 of file datastore_api.c.

372{
374
376 "Establishing DATASTORE connection!\n");
378 h->cfg = cfg;
380 if (NULL == h->mq)
381 {
382 GNUNET_free (h);
383 return NULL;
384 }
385 h->stats = GNUNET_STATISTICS_create ("datastore-api",
386 cfg);
387 return h;
388}
static void try_reconnect(void *cls)
Try reconnecting to the datastore service.
#define LOG(kind,...)
Definition: datastore_api.c:34
static struct GNUNET_ARM_Handle * h
Connection with ARM.
Definition: gnunet-arm.c:99
static struct GNUNET_CONFIGURATION_Handle * cfg
Our configuration.
Definition: gnunet-arm.c:109
@ GNUNET_ERROR_TYPE_DEBUG
#define GNUNET_new(type)
Allocate a struct or union of the given type.
#define GNUNET_free(ptr)
Wrapper around free.
struct GNUNET_STATISTICS_Handle * GNUNET_STATISTICS_create(const char *subsystem, const struct GNUNET_CONFIGURATION_Handle *cfg)
Get handle for the statistics service.
struct GNUNET_MQ_Handle * mq
Our connection to the ARM service.
Definition: arm_api.c:107
const struct GNUNET_CONFIGURATION_Handle * cfg
The configuration that we are using.
Definition: arm_api.c:112
Handle to the datastore service.

References cfg, GNUNET_ARM_Handle::cfg, GNUNET_ERROR_TYPE_DEBUG, GNUNET_free, GNUNET_new, GNUNET_STATISTICS_create(), h, LOG, GNUNET_ARM_Handle::mq, and try_reconnect().

Referenced by deserialize_publish_file(), GNUNET_FS_publish_ksk(), GNUNET_FS_publish_sks(), GNUNET_FS_publish_start(), GNUNET_FS_unindex_do_remove_(), GNUNET_FS_unindex_do_remove_kblocks_(), and run().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_disconnect()

void GNUNET_DATASTORE_disconnect ( struct GNUNET_DATASTORE_Handle h,
int  drop 
)

Disconnect from the datastore service (and free associated resources).

Parameters
hhandle to the datastore
dropset to GNUNET_YES to delete all data in datastore (!)

Definition at line 436 of file datastore_api.c.

438{
440
442 "Datastore disconnect\n");
443 if (NULL != h->mq)
444 {
446 h->mq = NULL;
447 }
448 if (NULL != h->reconnect_task)
449 {
451 h->reconnect_task = NULL;
452 }
453 while (NULL != (qe = h->queue_head))
454 {
455 switch (qe->response_type)
456 {
458 if (NULL != qe->qc.sc.cont)
459 qe->qc.sc.cont (qe->qc.sc.cont_cls,
462 _ ("Disconnected from DATASTORE"));
463 break;
464
466 if (NULL != qe->qc.rc.proc)
467 qe->qc.rc.proc (qe->qc.rc.proc_cls,
468 NULL,
469 0,
470 NULL,
471 0,
472 0,
473 0,
474 0,
476 0);
477 break;
478
479 default:
480 GNUNET_break (0);
481 }
483 }
484 if (GNUNET_YES == drop)
485 {
487 "Re-connecting to issue DROP!\n");
488 GNUNET_assert (NULL == h->mq);
490 "datastore",
491 NULL,
493 h);
494 if (NULL != h->mq)
495 {
496 struct GNUNET_MessageHeader *hdr;
497 struct GNUNET_MQ_Envelope *env;
498
499 env = GNUNET_MQ_msg (hdr,
503 h);
505 env);
506 return;
507 }
508 GNUNET_break (0);
509 }
511 GNUNET_NO);
512 h->stats = NULL;
513 GNUNET_free (h);
514}
struct GNUNET_MQ_Envelope * env
Definition: 005.c:1
static void disconnect_after_drop(void *cls)
Task used by to disconnect from the datastore after we send the GNUNET_MESSAGE_TYPE_DATASTORE_DROP me...
static void disconnect_on_mq_error(void *cls, enum GNUNET_MQ_Error error)
Handle error in sending drop request to datastore.
static void free_queue_entry(struct GNUNET_DATASTORE_QueueEntry *qe)
Free a queue entry.
static struct GNUNET_DATASTORE_QueueEntry * qe
Current operation.
struct GNUNET_MQ_Handle * GNUNET_CLIENT_connect(const struct GNUNET_CONFIGURATION_Handle *cfg, const char *service_name, const struct GNUNET_MQ_MessageHandler *handlers, GNUNET_MQ_ErrorHandler error_handler, void *error_handler_cls)
Create a message queue to connect to a GNUnet service.
Definition: client.c:1057
@ GNUNET_YES
@ GNUNET_NO
@ GNUNET_SYSERR
#define GNUNET_assert(cond)
Use this for fatal errors that cannot be handled.
#define GNUNET_break(cond)
Use this for internal assertion violations that are not fatal (can be handled) but should not occur.
void GNUNET_MQ_send(struct GNUNET_MQ_Handle *mq, struct GNUNET_MQ_Envelope *ev)
Send a message with the given message queue.
Definition: mq.c:304
#define GNUNET_MQ_msg(mvar, type)
Allocate a GNUNET_MQ_Envelope.
Definition: gnunet_mq_lib.h:78
void GNUNET_MQ_notify_sent(struct GNUNET_MQ_Envelope *ev, GNUNET_SCHEDULER_TaskCallback cb, void *cb_cls)
Call a callback once the envelope has been sent, that is, sending it can not be canceled anymore.
Definition: mq.c:638
void GNUNET_MQ_destroy(struct GNUNET_MQ_Handle *mq)
Destroy the message queue.
Definition: mq.c:683
#define GNUNET_MESSAGE_TYPE_DATASTORE_DROP
Message sent by datastore client to drop the database.
#define GNUNET_MESSAGE_TYPE_DATASTORE_DATA
Message sent by datastore to client providing requested data (in response to GET or GET_RANDOM reques...
#define GNUNET_MESSAGE_TYPE_DATASTORE_STATUS
Message sent by datastore to client informing about status processing a request (in response to RESER...
void * GNUNET_SCHEDULER_cancel(struct GNUNET_SCHEDULER_Task *task)
Cancel the task with the specified identifier.
Definition: scheduler.c:975
void GNUNET_STATISTICS_destroy(struct GNUNET_STATISTICS_Handle *h, int sync_first)
Destroy a handle (free all state associated with it).
#define GNUNET_TIME_UNIT_ZERO_ABS
Absolute time zero.
#define _(String)
GNU gettext support macro.
Definition: platform.h:178
struct GNUNET_SCHEDULER_Task * reconnect_task
ID of the reconnect task (if any).
Definition: arm_api.c:147
Entry in our priority queue.
union QueueContext qc
Context for the operation.
uint16_t response_type
Expected response type.
Header for all communications.
GNUNET_DATASTORE_DatumProcessor proc
Function to call with the result.
Definition: datastore_api.c:76
void * proc_cls
Closure for proc.
Definition: datastore_api.c:81
void * cont_cls
Closure for cont.
Definition: datastore_api.c:64
GNUNET_DATASTORE_ContinuationWithStatus cont
Continuation to call with the status.
Definition: datastore_api.c:59
struct ResultContext rc
Definition: datastore_api.c:92
struct StatusContext sc
Definition: datastore_api.c:90

References _, GNUNET_ARM_Handle::cfg, StatusContext::cont, StatusContext::cont_cls, disconnect_after_drop(), disconnect_on_mq_error(), env, free_queue_entry(), GNUNET_assert, GNUNET_break, GNUNET_CLIENT_connect(), GNUNET_ERROR_TYPE_DEBUG, GNUNET_free, GNUNET_MESSAGE_TYPE_DATASTORE_DATA, GNUNET_MESSAGE_TYPE_DATASTORE_DROP, GNUNET_MESSAGE_TYPE_DATASTORE_STATUS, GNUNET_MQ_destroy(), GNUNET_MQ_msg, GNUNET_MQ_notify_sent(), GNUNET_MQ_send(), GNUNET_NO, GNUNET_SCHEDULER_cancel(), GNUNET_STATISTICS_destroy(), GNUNET_SYSERR, GNUNET_TIME_UNIT_ZERO_ABS, GNUNET_YES, h, LOG, GNUNET_ARM_Handle::mq, ResultContext::proc, ResultContext::proc_cls, GNUNET_DATASTORE_QueueEntry::qc, qe, QueueContext::rc, GNUNET_ARM_Handle::reconnect_task, GNUNET_DATASTORE_QueueEntry::response_type, and QueueContext::sc.

Referenced by disconnect_after_drop(), disconnect_on_mq_error(), do_shutdown(), GNUNET_FS_publish_ksk_cancel(), GNUNET_FS_publish_sks_cancel(), GNUNET_FS_unindex_do_remove_(), GNUNET_FS_unindex_signal_suspend_(), GNUNET_FS_unindex_stop(), publish_cleanup(), shutdown_task(), suspend_operation(), and unindex_finish().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_reserve()

struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_reserve ( struct GNUNET_DATASTORE_Handle h,
uint64_t  amount,
uint32_t  entries,
GNUNET_DATASTORE_ContinuationWithStatus  cont,
void *  cont_cls 
)

Reserve space in the datastore.

This function should be used to avoid "out of space" failures during a longer sequence of "put" operations (for example, when a file is being inserted).

Parameters
hhandle to the datastore
amounthow much space (in bytes) should be reserved (for content only)
entrieshow many entries will be created (to calculate per-entry overhead)
contcontinuation to call when done; "success" will be set to a positive reservation value if space could be reserved.
cont_clsclosure for cont
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel; note that even if NULL is returned, the callback will be invoked (or rather, will already have been invoked)

Definition at line 1031 of file datastore_api.c.

1036{
1038 struct GNUNET_MQ_Envelope *env;
1039 struct ReserveMessage *rm;
1040 union QueueContext qc;
1041
1042 if (NULL == cont)
1043 cont = &drop_status_cont;
1045 "Asked to reserve %llu bytes of data and %u entries\n",
1046 (unsigned long long) amount,
1047 (unsigned int) entries);
1048 env = GNUNET_MQ_msg (rm,
1050 rm->entries = htonl (entries);
1051 rm->amount = GNUNET_htonll (amount);
1052
1053 qc.sc.cont = cont;
1054 qc.sc.cont_cls = cont_cls;
1056 env,
1057 UINT_MAX,
1058 UINT_MAX,
1060 &qc);
1061 if (NULL == qe)
1062 {
1064 "Could not create queue entry to reserve\n");
1065 return NULL;
1066 }
1068 gettext_noop ("# RESERVE requests executed"),
1069 1,
1070 GNUNET_NO);
1071 process_queue (h);
1072 return qe;
1073}
static void process_queue(struct GNUNET_DATASTORE_Handle *h)
Process entries in the queue (or do nothing if we are already doing so).
static struct GNUNET_DATASTORE_QueueEntry * make_queue_entry(struct GNUNET_DATASTORE_Handle *h, struct GNUNET_MQ_Envelope *env, unsigned int queue_priority, unsigned int max_queue_size, uint16_t expected_type, const union QueueContext *qc)
Create a new entry for our priority queue (and possibly discard other entries if the queue is getting...
static void drop_status_cont(void *cls, int32_t result, struct GNUNET_TIME_Absolute min_expiration, const char *emsg)
Dummy continuation used to do nothing (but be non-zero).
#define gettext_noop(String)
Definition: gettext.h:70
uint64_t GNUNET_htonll(uint64_t n)
Convert unsigned 64-bit integer to network byte order.
Definition: common_endian.c:37
#define GNUNET_MESSAGE_TYPE_DATASTORE_RESERVE
Message sent by datastore client on join.
void GNUNET_STATISTICS_update(struct GNUNET_STATISTICS_Handle *handle, const char *name, int64_t delta, int make_persistent)
Set statistic value for the peer.
Message from datastore service informing client about the current size of the datastore.
Definition: datastore.h:40
uint32_t entries
Number of items to reserve.
Definition: datastore.h:49
uint64_t amount
Number of bytes to reserve.
Definition: datastore.h:54
Context for a queue operation.
Definition: datastore_api.c:89

References ReserveMessage::amount, StatusContext::cont, StatusContext::cont_cls, drop_status_cont(), ReserveMessage::entries, env, gettext_noop, GNUNET_ERROR_TYPE_DEBUG, GNUNET_htonll(), GNUNET_MESSAGE_TYPE_DATASTORE_RESERVE, GNUNET_MESSAGE_TYPE_DATASTORE_STATUS, GNUNET_MQ_msg, GNUNET_NO, GNUNET_STATISTICS_update(), h, LOG, make_queue_entry(), process_queue(), qe, and QueueContext::sc.

Referenced by GNUNET_FS_publish_start().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_put()

struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_put ( struct GNUNET_DATASTORE_Handle h,
uint32_t  rid,
const struct GNUNET_HashCode key,
size_t  size,
const void *  data,
enum GNUNET_BLOCK_Type  type,
uint32_t  priority,
uint32_t  anonymity,
uint32_t  replication,
struct GNUNET_TIME_Absolute  expiration,
unsigned int  queue_priority,
unsigned int  max_queue_size,
GNUNET_DATASTORE_ContinuationWithStatus  cont,
void *  cont_cls 
)

Store an item in the datastore.

If the item is already present, the priorities and replication values are summed up and the higher expiration time and lower anonymity level is used.

Parameters
hhandle to the datastore
ridreservation ID to use (from "reserve"); use 0 if no prior reservation was made
keykey for the value
sizenumber of bytes in data
datacontent stored
typetype of the content
prioritypriority of the content
anonymityanonymity-level for the content
replicationhow often should the content be replicated to other peers?
expirationexpiration time for the content
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
contcontinuation to call when done
cont_clsclosure for cont
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel; note that even if NULL is returned, the callback will be invoked (or rather, will already have been invoked)

Definition at line 945 of file datastore_api.c.

959{
961 struct GNUNET_MQ_Envelope *env;
962 struct DataMessage *dm;
963 union QueueContext qc;
964
965 if (size + sizeof(*dm) >= GNUNET_MAX_MESSAGE_SIZE)
966 {
967 GNUNET_break (0);
968 return NULL;
969 }
970
972 "Asked to put %lu bytes of data under key `%s' for %s\n",
973 (unsigned long) size,
974 GNUNET_h2s (key),
977 GNUNET_YES));
979 size,
981 dm->rid = htonl (rid);
982 dm->size = htonl ((uint32_t) size);
983 dm->type = htonl (type);
984 dm->priority = htonl (priority);
985 dm->anonymity = htonl (anonymity);
986 dm->replication = htonl (replication);
988 dm->key = *key;
989 GNUNET_memcpy (&dm[1],
990 data,
991 size);
992 qc.sc.cont = cont;
993 qc.sc.cont_cls = cont_cls;
995 env,
996 queue_priority,
997 max_queue_size,
999 &qc);
1000 if (NULL == qe)
1001 {
1003 "Could not create queue entry for PUT\n");
1004 return NULL;
1005 }
1007 gettext_noop ("# PUT requests executed"),
1008 1,
1009 GNUNET_NO);
1010 process_queue (h);
1011 return qe;
1012}
static unsigned int replication
Desired replication level.
static char * data
The data to insert into the dht.
struct GNUNET_HashCode key
The key used in the DHT.
static struct GNUNET_TIME_Relative expiration
User supplied expiration value.
static unsigned int anonymity
static uint32_t type
Type string converted to DNS type value.
#define GNUNET_MAX_MESSAGE_SIZE
Largest supported message (to be precise, one byte more than the largest possible message,...
#define GNUNET_memcpy(dst, src, n)
Call memcpy() but check for n being 0 first.
const char * GNUNET_h2s(const struct GNUNET_HashCode *hc)
Convert a hash value to a string (for printing debug messages).
#define GNUNET_MQ_msg_extra(mvar, esize, type)
Allocate an envelope, with extra space allocated after the space needed by the message struct.
Definition: gnunet_mq_lib.h:63
#define GNUNET_MESSAGE_TYPE_DATASTORE_PUT
Message sent by datastore client to store data.
struct GNUNET_TIME_Relative GNUNET_TIME_absolute_get_remaining(struct GNUNET_TIME_Absolute future)
Given a timestamp in the future, how much time remains until then?
Definition: time.c:405
const char * GNUNET_STRINGS_relative_time_to_string(struct GNUNET_TIME_Relative delta, int do_round)
Give relative time in human-readable fancy format.
Definition: strings.c:570
struct GNUNET_TIME_AbsoluteNBO GNUNET_TIME_absolute_hton(struct GNUNET_TIME_Absolute a)
Convert absolute time to network byte order.
Definition: time.c:638
static unsigned int size
Size of the "table".
Definition: peer.c:68
Message transmitting content from or to the datastore service.
Definition: datastore.h:192
uint32_t priority
Priority of the item (NBO), zero for remove.
Definition: datastore.h:219
struct GNUNET_HashCode key
Key under which the item can be found.
Definition: datastore.h:252
struct GNUNET_TIME_AbsoluteNBO expiration
Expiration time (NBO); zero for remove.
Definition: datastore.h:247
uint32_t type
Type of the item (NBO), zero for remove, (actually an enum GNUNET_BLOCK_Type)
Definition: datastore.h:214
uint32_t size
Number of bytes in the item (NBO).
Definition: datastore.h:209
uint32_t replication
Desired replication level.
Definition: datastore.h:229
uint32_t anonymity
Desired anonymity level (NBO), zero for remove.
Definition: datastore.h:224
uint32_t rid
Reservation ID to use; use zero for none.
Definition: datastore.h:204

References anonymity, DataMessage::anonymity, StatusContext::cont, StatusContext::cont_cls, data, env, expiration, DataMessage::expiration, gettext_noop, GNUNET_break, GNUNET_ERROR_TYPE_DEBUG, GNUNET_h2s(), GNUNET_MAX_MESSAGE_SIZE, GNUNET_memcpy, GNUNET_MESSAGE_TYPE_DATASTORE_PUT, GNUNET_MESSAGE_TYPE_DATASTORE_STATUS, GNUNET_MQ_msg_extra, GNUNET_NO, GNUNET_STATISTICS_update(), GNUNET_STRINGS_relative_time_to_string(), GNUNET_TIME_absolute_get_remaining(), GNUNET_TIME_absolute_hton(), GNUNET_YES, h, key, DataMessage::key, LOG, make_queue_entry(), DataMessage::priority, process_queue(), qe, replication, DataMessage::replication, DataMessage::rid, QueueContext::sc, size, DataMessage::size, type, and DataMessage::type.

Referenced by block_proc(), GNUNET_FS_publish_ublock_(), handle_dht_reply(), handle_p2p_put(), and put_cb().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_release_reserve()

struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_release_reserve ( struct GNUNET_DATASTORE_Handle h,
uint32_t  rid,
unsigned int  queue_priority,
unsigned int  max_queue_size,
GNUNET_DATASTORE_ContinuationWithStatus  cont,
void *  cont_cls 
)

Signal that all of the data for which a reservation was made has been stored and that whatever excess space might have been reserved can now be released.

Parameters
hhandle to the datastore
ridreservation ID (value of "success" in original continuation from the "reserve" function).
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
contcontinuation to call when done
cont_clsclosure for cont
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel; note that even if NULL is returned, the callback will be invoked (or rather, will already have been invoked)

Definition at line 1077 of file datastore_api.c.

1083{
1085 struct GNUNET_MQ_Envelope *env;
1086 struct ReleaseReserveMessage *rrm;
1087 union QueueContext qc;
1088
1089 if (NULL == cont)
1090 cont = &drop_status_cont;
1092 "Asked to release reserve %d\n",
1093 rid);
1094 env = GNUNET_MQ_msg (rrm,
1096 rrm->rid = htonl (rid);
1097 qc.sc.cont = cont;
1098 qc.sc.cont_cls = cont_cls;
1100 env,
1101 queue_priority,
1102 max_queue_size,
1104 &qc);
1105 if (NULL == qe)
1106 {
1108 "Could not create queue entry to release reserve\n");
1109 return NULL;
1110 }
1113 ("# RELEASE RESERVE requests executed"), 1,
1114 GNUNET_NO);
1115 process_queue (h);
1116 return qe;
1117}
#define GNUNET_MESSAGE_TYPE_DATASTORE_RELEASE_RESERVE
Message sent by datastore client on join.
Message from datastore client informing service that the remainder of the reserved bytes can now be r...
Definition: datastore.h:90
int32_t rid
Reservation id.
Definition: datastore.h:99

References StatusContext::cont, StatusContext::cont_cls, drop_status_cont(), env, gettext_noop, GNUNET_ERROR_TYPE_DEBUG, GNUNET_MESSAGE_TYPE_DATASTORE_RELEASE_RESERVE, GNUNET_MESSAGE_TYPE_DATASTORE_STATUS, GNUNET_MQ_msg, GNUNET_NO, GNUNET_STATISTICS_update(), h, LOG, make_queue_entry(), process_queue(), qe, ReleaseReserveMessage::rid, and QueueContext::sc.

Referenced by publish_sblocks_cont().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_remove()

struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_remove ( struct GNUNET_DATASTORE_Handle h,
const struct GNUNET_HashCode key,
size_t  size,
const void *  data,
unsigned int  queue_priority,
unsigned int  max_queue_size,
GNUNET_DATASTORE_ContinuationWithStatus  cont,
void *  cont_cls 
)

Explicitly remove some content from the database.

cont will be called with status GNUNET_OK if content was removed, GNUNET_NO if no matching entry was found and GNUNET_SYSERR on all other types of errors.

Parameters
hhandle to the datastore
keykey for the value
sizenumber of bytes in data
datacontent stored
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
contcontinuation to call when done
cont_clsclosure for cont
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel; note that even if NULL is returned, the callback will be invoked (or rather, will already have been invoked)

Definition at line 1121 of file datastore_api.c.

1129{
1131 struct DataMessage *dm;
1132 struct GNUNET_MQ_Envelope *env;
1133 union QueueContext qc;
1134
1135 if (sizeof(*dm) + size >= GNUNET_MAX_MESSAGE_SIZE)
1136 {
1137 GNUNET_break (0);
1138 return NULL;
1139 }
1140 if (NULL == cont)
1141 cont = &drop_status_cont;
1143 "Asked to remove %lu bytes under key `%s'\n",
1144 (unsigned long) size,
1145 GNUNET_h2s (key));
1147 size,
1149 dm->size = htonl (size);
1150 dm->key = *key;
1151 GNUNET_memcpy (&dm[1],
1152 data,
1153 size);
1154
1155 qc.sc.cont = cont;
1156 qc.sc.cont_cls = cont_cls;
1157
1159 env,
1160 queue_priority,
1161 max_queue_size,
1163 &qc);
1164 if (NULL == qe)
1165 {
1167 "Could not create queue entry for REMOVE\n");
1168 return NULL;
1169 }
1171 gettext_noop ("# REMOVE requests executed"),
1172 1,
1173 GNUNET_NO);
1174 process_queue (h);
1175 return qe;
1176}
#define GNUNET_MESSAGE_TYPE_DATASTORE_REMOVE
Message sent by datastore client to remove data.

References StatusContext::cont, StatusContext::cont_cls, data, drop_status_cont(), env, gettext_noop, GNUNET_break, GNUNET_ERROR_TYPE_DEBUG, GNUNET_h2s(), GNUNET_MAX_MESSAGE_SIZE, GNUNET_memcpy, GNUNET_MESSAGE_TYPE_DATASTORE_REMOVE, GNUNET_MESSAGE_TYPE_DATASTORE_STATUS, GNUNET_MQ_msg_extra, GNUNET_NO, GNUNET_STATISTICS_update(), h, key, DataMessage::key, LOG, make_queue_entry(), process_queue(), qe, QueueContext::sc, size, and DataMessage::size.

Referenced by GNUNET_FS_handle_on_demand_block(), process_kblock_for_unindex(), process_local_reply(), and unindex_process().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_get_key()

struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_get_key ( struct GNUNET_DATASTORE_Handle h,
uint64_t  next_uid,
bool  random,
const struct GNUNET_HashCode key,
enum GNUNET_BLOCK_Type  type,
unsigned int  queue_priority,
unsigned int  max_queue_size,
GNUNET_DATASTORE_DatumProcessor  proc,
void *  proc_cls 
)

Get a result for a particular key from the datastore.

The processor will only be called once.

Parameters
hhandle to the datastore
next_uidreturn the result with lowest uid >= next_uid
randomif true, return a random result instead of using next_uid
keymaybe NULL (to match all entries)
typedesired type, 0 for any
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
procfunction to call on a matching value; or with a NULL value if no datum matches
proc_clsclosure for proc
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel

The processor will only be called once.

Parameters
hhandle to the datastore
next_uidreturn the result with lowest uid >= next_uid
randomif true, return a random result instead of using next_uid
keymaybe NULL (to match all entries)
typedesired type, 0 for any
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
procfunction to call on each matching value; will be called once with a NULL value at the end
proc_clsclosure for proc
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel

Definition at line 1302 of file datastore_api.c.

1311{
1313 struct GNUNET_MQ_Envelope *env;
1314 struct GetKeyMessage *gkm;
1315 struct GetMessage *gm;
1316 union QueueContext qc;
1317
1318 GNUNET_assert (NULL != proc);
1320 "Asked to look for data of type %u under key `%s'\n",
1321 (unsigned int) type,
1322 (NULL != key) ? GNUNET_h2s (key) : "NULL");
1323 if (NULL == key)
1324 {
1325 env = GNUNET_MQ_msg (gm,
1327 gm->type = htonl (type);
1328 gm->next_uid = GNUNET_htonll (next_uid);
1329 gm->random = random;
1330 }
1331 else
1332 {
1333 env = GNUNET_MQ_msg (gkm,
1335 gkm->type = htonl (type);
1336 gkm->next_uid = GNUNET_htonll (next_uid);
1337 gkm->random = random;
1338 gkm->key = *key;
1339 }
1340 qc.rc.proc = proc;
1341 qc.rc.proc_cls = proc_cls;
1343 env,
1344 queue_priority,
1345 max_queue_size,
1347 &qc);
1348 if (NULL == qe)
1349 {
1351 "Could not queue request for `%s'\n",
1352 (NULL != key) ? GNUNET_h2s (key): "NULL");
1353 return NULL;
1354 }
1355#if INSANE_STATISTICS
1357 gettext_noop ("# GET requests executed"),
1358 1,
1359 GNUNET_NO);
1360#endif
1361 process_queue (h);
1362 return qe;
1363}
#define GNUNET_MESSAGE_TYPE_DATASTORE_GET_KEY
Message sent by datastore client to get data by key.
#define GNUNET_MESSAGE_TYPE_DATASTORE_GET
Message sent by datastore client to get data.
Message to the datastore service asking about specific content.
Definition: datastore.h:108
uint32_t type
Desired content type.
Definition: datastore.h:117
uint64_t next_uid
UID at which to start the search.
Definition: datastore.h:122
struct GNUNET_HashCode key
Desired key.
Definition: datastore.h:132
uint32_t random
If true return a random result.
Definition: datastore.h:127
Message to the datastore service asking about specific content.
Definition: datastore.h:141
uint32_t random
If true return a random result.
Definition: datastore.h:160
uint32_t type
Desired content type.
Definition: datastore.h:150
uint64_t next_uid
UID at which to start the search.
Definition: datastore.h:155

References env, gettext_noop, GNUNET_assert, GNUNET_ERROR_TYPE_DEBUG, GNUNET_h2s(), GNUNET_htonll(), GNUNET_MESSAGE_TYPE_DATASTORE_DATA, GNUNET_MESSAGE_TYPE_DATASTORE_GET, GNUNET_MESSAGE_TYPE_DATASTORE_GET_KEY, GNUNET_MQ_msg, GNUNET_NO, GNUNET_STATISTICS_update(), h, key, GetKeyMessage::key, LOG, make_queue_entry(), GetKeyMessage::next_uid, GetMessage::next_uid, ResultContext::proc, ResultContext::proc_cls, process_queue(), qe, GetKeyMessage::random, GetMessage::random, QueueContext::rc, type, GetKeyMessage::type, and GetMessage::type.

Referenced by do_get(), GNUNET_FS_unindex_do_remove_kblocks_(), handle_request(), process_kblock_for_unindex(), and start_local_query().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_get_zero_anonymity()

struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_get_zero_anonymity ( struct GNUNET_DATASTORE_Handle h,
uint64_t  next_uid,
unsigned int  queue_priority,
unsigned int  max_queue_size,
enum GNUNET_BLOCK_Type  type,
GNUNET_DATASTORE_DatumProcessor  proc,
void *  proc_cls 
)

Get a single zero-anonymity value from the datastore.

Parameters
hhandle to the datastore
next_uidreturn the result with lowest uid >= next_uid
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
typeallowed type for the operation (never zero)
procfunction to call on a random value; it will be called once with a value (if available) or with NULL if none value exists.
proc_clsclosure for proc
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel

Definition at line 1238 of file datastore_api.c.

1245{
1247 struct GNUNET_MQ_Envelope *env;
1248 struct GetZeroAnonymityMessage *m;
1249 union QueueContext qc;
1250
1251 GNUNET_assert (NULL != proc);
1254 "Asked to get a zero-anonymity entry of type %d\n",
1255 type);
1256 env = GNUNET_MQ_msg (m,
1258 m->type = htonl ((uint32_t) type);
1259 m->next_uid = GNUNET_htonll (next_uid);
1260 qc.rc.proc = proc;
1261 qc.rc.proc_cls = proc_cls;
1263 env,
1264 queue_priority,
1265 max_queue_size,
1267 &qc);
1268 if (NULL == qe)
1269 {
1271 "Could not create queue entry for zero-anonymity procation\n");
1272 return NULL;
1273 }
1276 ("# GET ZERO ANONYMITY requests executed"), 1,
1277 GNUNET_NO);
1278 process_queue (h);
1279 return qe;
1280}
static struct GNUNET_ARM_MonitorHandle * m
Monitor connection with ARM.
Definition: gnunet-arm.c:104
#define GNUNET_MESSAGE_TYPE_DATASTORE_GET_ZERO_ANONYMITY
Message sent by datastore client to get random data.
@ GNUNET_BLOCK_TYPE_ANY
Identifier for any block.
Message to the datastore service asking about zero anonymity content.
Definition: datastore.h:169

References env, gettext_noop, GNUNET_assert, GNUNET_BLOCK_TYPE_ANY, GNUNET_ERROR_TYPE_DEBUG, GNUNET_htonll(), GNUNET_MESSAGE_TYPE_DATASTORE_DATA, GNUNET_MESSAGE_TYPE_DATASTORE_GET_ZERO_ANONYMITY, GNUNET_MQ_msg, GNUNET_NO, GNUNET_STATISTICS_update(), h, LOG, m, make_queue_entry(), ResultContext::proc, ResultContext::proc_cls, process_queue(), qe, QueueContext::rc, and type.

Referenced by gather_dht_put_blocks().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_get_for_replication()

struct GNUNET_DATASTORE_QueueEntry * GNUNET_DATASTORE_get_for_replication ( struct GNUNET_DATASTORE_Handle h,
unsigned int  queue_priority,
unsigned int  max_queue_size,
GNUNET_DATASTORE_DatumProcessor  proc,
void *  proc_cls 
)

Get a random value from the datastore for content replication.

Returns a single, random value among those with the highest replication score, lowering positive replication scores by one for the chosen value (if only content with a replication score exists, a random value is returned and replication scores are not changed).

Parameters
hhandle to the datastore
queue_priorityranking of this request in the priority queue
max_queue_sizeat what queue size should this request be dropped (if other requests of higher priority are in the queue)
procfunction to call on a random value; it will be called once with a value (if available) and always once with a value of NULL.
proc_clsclosure for proc
Returns
NULL if the entry was not queued, otherwise a handle that can be used to cancel

Definition at line 1198 of file datastore_api.c.

1203{
1205 struct GNUNET_MQ_Envelope *env;
1206 struct GNUNET_MessageHeader *m;
1207 union QueueContext qc;
1208
1209 GNUNET_assert (NULL != proc);
1211 "Asked to get replication entry\n");
1212 env = GNUNET_MQ_msg (m,
1214 qc.rc.proc = proc;
1215 qc.rc.proc_cls = proc_cls;
1217 env,
1218 queue_priority,
1219 max_queue_size,
1221 &qc);
1222 if (NULL == qe)
1223 {
1225 "Could not create queue entry for GET REPLICATION\n");
1226 return NULL;
1227 }
1230 ("# GET REPLICATION requests executed"), 1,
1231 GNUNET_NO);
1232 process_queue (h);
1233 return qe;
1234}
#define GNUNET_MESSAGE_TYPE_DATASTORE_GET_REPLICATION
Message sent by datastore client to get random data.

References env, gettext_noop, GNUNET_assert, GNUNET_ERROR_TYPE_DEBUG, GNUNET_MESSAGE_TYPE_DATASTORE_DATA, GNUNET_MESSAGE_TYPE_DATASTORE_GET_REPLICATION, GNUNET_MQ_msg, GNUNET_NO, GNUNET_STATISTICS_update(), h, LOG, m, make_queue_entry(), ResultContext::proc, ResultContext::proc_cls, process_queue(), qe, and QueueContext::rc.

Referenced by gather_migration_blocks().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ GNUNET_DATASTORE_cancel()

void GNUNET_DATASTORE_cancel ( struct GNUNET_DATASTORE_QueueEntry qe)

Cancel a datastore operation.

The final callback from the operation must not have been done yet.

Parameters
qeoperation to cancel

Definition at line 1373 of file datastore_api.c.

1374{
1375 struct GNUNET_DATASTORE_Handle *h = qe->h;
1376
1378 "Pending DATASTORE request %p cancelled (%d, %d)\n",
1379 qe,
1380 NULL == qe->env,
1381 h->queue_head == qe);
1382 if (NULL == qe->env)
1383 {
1385 h->skip_next_messages++;
1386 return;
1387 }
1389 process_queue (h);
1390}
struct GNUNET_DATASTORE_Handle * h
Handle to the master context.
struct GNUNET_MQ_Envelope * env
Envelope of the request to transmit, NULL after transmission.

References GNUNET_DATASTORE_QueueEntry::env, free_queue_entry(), GNUNET_ERROR_TYPE_DEBUG, h, GNUNET_DATASTORE_QueueEntry::h, LOG, process_queue(), and qe.

Referenced by clean_request(), disconnect_cb(), do_shutdown(), GNUNET_FS_publish_stop(), GNUNET_FS_publish_ublock_cancel_(), GNUNET_FS_unindex_signal_suspend_(), GNUNET_FS_unindex_stop(), GSF_pending_request_cancel_(), GSF_push_done_(), GSF_put_done_(), and suspend_operation().

Here is the call graph for this function:
Here is the caller graph for this function: