Block group library

Library for data group management. More...

Collaboration diagram for Block group library:

Enumerations
enum	GNUNET_BLOCK_ReplyEvaluationResult {   GNUNET_BLOCK_REPLY_TYPE_NOT_SUPPORTED = -1 , GNUNET_BLOCK_REPLY_OK_DUPLICATE = 0 , GNUNET_BLOCK_REPLY_IRRELEVANT = 1 , GNUNET_BLOCK_REPLY_OK_MORE = 2 ,   GNUNET_BLOCK_REPLY_OK_LAST = 3 }
	Possible ways for how a block may relate to a query. More...

Functions
size_t	GNUNET_BLOCK_GROUP_compute_bloomfilter_size (unsigned int entry_count, unsigned int k)
	How many bytes should a bloomfilter be if we have already seen entry_count responses? Sized so that do not have to re-size the filter too often (to keep it cheap). More...
struct GNUNET_BLOCK_Group *	GNUNET_BLOCK_GROUP_bf_create (void *cls, size_t bf_size, unsigned int bf_k, enum GNUNET_BLOCK_Type type, const void *raw_data, size_t raw_data_size)
	Create a new block group that filters duplicates using a Bloom filter. More...
enum GNUNET_GenericReturnValue	GNUNET_BLOCK_GROUP_bf_test_and_set (struct GNUNET_BLOCK_Group *bg, const struct GNUNET_HashCode *hc)
	Test if hc is contained in the Bloom filter of bg. More...
void	GNUNET_BLOCK_mingle_hash (const struct GNUNET_HashCode *in, uint32_t mingle_number, struct GNUNET_HashCode *hc)
	Mingle hash with the mingle_number to produce different bits. More...
struct GNUNET_BLOCK_Context *	GNUNET_BLOCK_context_create (const struct GNUNET_CONFIGURATION_Handle *cfg)
	Create a block context. More...
void	GNUNET_BLOCK_context_destroy (struct GNUNET_BLOCK_Context *ctx)
	Destroy the block context. More...
struct GNUNET_BLOCK_Group *	GNUNET_BLOCK_group_create (struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, const void *raw_data, size_t raw_data_size,...)
	Create a new block group. More...
enum GNUNET_GenericReturnValue	GNUNET_BLOCK_group_serialize (struct GNUNET_BLOCK_Group *bg, void **raw_data, size_t *raw_data_size)
	Serialize state of a block group. More...
void	GNUNET_BLOCK_group_destroy (struct GNUNET_BLOCK_Group *bg)
	Destroy resources used by a block group. More...
enum GNUNET_BLOCK_ReplyEvaluationResult	GNUNET_BLOCK_check_reply (struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, struct GNUNET_BLOCK_Group *group, const struct GNUNET_HashCode *query, const void *xquery, size_t xquery_size, const void *reply_block, size_t reply_block_size)
	Function called to validate if a reply is good for a particular query. More...
enum GNUNET_GenericReturnValue	GNUNET_BLOCK_check_query (struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, const struct GNUNET_HashCode *query, const void *xquery, size_t xquery_size)
	Function called to validate a request. More...
enum GNUNET_GenericReturnValue	GNUNET_BLOCK_check_block (struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, const void *block, size_t block_size)
	Function called to validate a block. More...
enum GNUNET_GenericReturnValue	GNUNET_BLOCK_get_key (struct GNUNET_BLOCK_Context *ctx, enum GNUNET_BLOCK_Type type, const void *block, size_t block_size, struct GNUNET_HashCode *key)
	Function called to obtain the key for a block. More...
enum GNUNET_GenericReturnValue	GNUNET_BLOCK_group_set_seen (struct GNUNET_BLOCK_Group *bg, const struct GNUNET_HashCode *seen_results, unsigned int seen_results_count)
	Update block group to filter out the given results. More...
enum GNUNET_GenericReturnValue	GNUNET_BLOCK_group_merge (struct GNUNET_BLOCK_Group *bg1, struct GNUNET_BLOCK_Group *bg2)
	Try merging two block groups. More...

Detailed Description

Library for data group management.

Library for data block manipulation.

Enumeration Type Documentation

GNUNET_BLOCK_ReplyEvaluationResult

enum GNUNET_BLOCK_ReplyEvaluationResult

Possible ways for how a block may relate to a query.

Enumerator
GNUNET_BLOCK_REPLY_TYPE_NOT_SUPPORTED	Specified block type not supported by any plugin.
GNUNET_BLOCK_REPLY_OK_DUPLICATE	Valid result, but suppressed because it is a duplicate.
GNUNET_BLOCK_REPLY_IRRELEVANT	Block does not match xquery (valid result, not relevant for the request)
GNUNET_BLOCK_REPLY_OK_MORE	Valid result, and there may be more.
GNUNET_BLOCK_REPLY_OK_LAST	Last possible valid result.

Definition at line 53 of file gnunet_block_lib.h.

{
 
  GNUNET_BLOCK_REPLY_TYPE_NOT_SUPPORTED = -1,
 
  GNUNET_BLOCK_REPLY_OK_DUPLICATE = 0,
 
  GNUNET_BLOCK_REPLY_IRRELEVANT = 1,
 
  GNUNET_BLOCK_REPLY_OK_MORE = 2,
 
  GNUNET_BLOCK_REPLY_OK_LAST = 3
 
};

Function Documentation

GNUNET_BLOCK_GROUP_compute_bloomfilter_size()

size_t GNUNET_BLOCK_GROUP_compute_bloomfilter_size	(	unsigned int	entry_count,
		unsigned int	k
	)

How many bytes should a bloomfilter be if we have already seen entry_count responses? Sized so that do not have to re-size the filter too often (to keep it cheap).

Since other peers will also add entries but not resize the filter, we should generally pick a slightly larger size than what the strict math would suggest.

Parameters

entry_count	expected number of entries in the Bloom filter
k	number of bits set per entry

Returns

must be a power of two and smaller or equal to 2^15.

Definition at line 268 of file bg_bf.c.

{
  size_t size;
  unsigned int ideal = (entry_count * k) / 4;
  uint16_t max = 1 << 15;
 
  if (entry_count > max)
    return max;
  size = 8;
  while ((size < max) && (size < ideal))
    size *= 2;
  if (size > max)
    return max;
  return size;
}

References max, and size.

Referenced by block_plugin_dht_check_block(), block_plugin_dns_check_block(), block_plugin_fs_check_block(), block_plugin_gns_check_block(), block_plugin_regex_check_block(), block_plugin_template_check_block(), and block_plugin_test_check_block().

Here is the caller graph for this function:

GNUNET_BLOCK_GROUP_bf_create()

struct GNUNET_BLOCK_Group* GNUNET_BLOCK_GROUP_bf_create	(	void *	cls,
		size_t	bf_size,
		unsigned int	bf_k,
		enum GNUNET_BLOCK_Type	type,
		const void *	raw_data,
		size_t	raw_data_size
	)

Create a new block group that filters duplicates using a Bloom filter.

Parameters

ctx	block context in which the block group is created
bf_size	size of the Bloom filter
bf_k	K-value for the Bloom filter
type	block type
raw_data	optional serialized prior state of the group, NULL if unavailable/fresh
raw_data_size	number of bytes in raw_data, 0 if unavailable/fresh

Returns

block group handle, NULL if block groups are not supported by this type of block (this is not an error)

Definition at line 173 of file bg_bf.c.

{
  struct BfGroupInternals *gi;
  struct GNUNET_BLOCK_Group *bg;
  uint32_t nonce;
 
  if ( (NULL != raw_data) &&
       (raw_data_size < sizeof (nonce)) )
  {
    GNUNET_break_op (0);
    return NULL;
  }
  if (NULL != raw_data)
  {
    memcpy (&nonce,
            raw_data,
            sizeof (nonce));
    raw_data += sizeof (nonce);
    raw_data_size -= sizeof (nonce);
  }
  else
  {
    nonce = GNUNET_CRYPTO_random_u32 (GNUNET_CRYPTO_QUALITY_NONCE,
                                      UINT32_MAX);
  }
  gi = GNUNET_new (struct BfGroupInternals);
  gi->bf = GNUNET_CONTAINER_bloomfilter_init ((bf_size != raw_data_size) ?
                                              NULL : raw_data,
                                              bf_size,
                                              bf_k);
  gi->bf_mutator = nonce;
  gi->bf_size = bf_size;
  bg = GNUNET_new (struct GNUNET_BLOCK_Group);
  bg->type = type;
  bg->serialize_cb = &bf_group_serialize_cb;
  bg->mark_seen_cb = &bf_group_mark_seen_cb;
  bg->merge_cb = &bf_group_merge_cb;
  bg->destroy_cb = &bf_group_destroy_cb;
  bg->internal_cls = gi;
  return bg;
}

References BfGroupInternals::bf, bf_group_destroy_cb(), bf_group_mark_seen_cb(), bf_group_merge_cb(), bf_group_serialize_cb(), BfGroupInternals::bf_mutator, BfGroupInternals::bf_size, GNUNET_BLOCK_Group::destroy_cb, GNUNET_break_op, GNUNET_CONTAINER_bloomfilter_init(), GNUNET_CRYPTO_QUALITY_NONCE, GNUNET_CRYPTO_random_u32(), GNUNET_new, GNUNET_BLOCK_Group::internal_cls, GNUNET_BLOCK_Group::mark_seen_cb, GNUNET_BLOCK_Group::merge_cb, GNUNET_BLOCK_Group::serialize_cb, GNUNET_BLOCK_Group::type, and type.

Referenced by block_plugin_dht_check_block(), block_plugin_dns_check_block(), block_plugin_fs_check_block(), block_plugin_gns_check_block(), block_plugin_regex_check_block(), block_plugin_template_check_block(), and block_plugin_test_check_block().

Here is the call graph for this function:

Here is the caller graph for this function:

GNUNET_BLOCK_GROUP_bf_test_and_set()

enum GNUNET_GenericReturnValue GNUNET_BLOCK_GROUP_bf_test_and_set	(	struct GNUNET_BLOCK_Group *	bg,
		const struct GNUNET_HashCode *	hc
	)

Test if hc is contained in the Bloom filter of bg.

If so, return GNUNET_YES. If not, add hc to the Bloom filter and return GNUNET_NO.

Parameters

bg	block group to use for testing
hc	hash of element to evaluate

Returns

GNUNET_YES if hc is (likely) a duplicate GNUNET_NO if hc was definitively not in bg (but now is)

Definition at line 173 of file bg_bf.c.

{
  struct BfGroupInternals *gi;
  struct GNUNET_HashCode mhash;
 
  if (NULL == bg)
    return GNUNET_NO;
  gi = bg->internal_cls;
  GNUNET_BLOCK_mingle_hash (hc,
                            gi->bf_mutator,
                            &mhash);
  if (GNUNET_YES ==
      GNUNET_CONTAINER_bloomfilter_test (gi->bf,
                                         &mhash))
    return GNUNET_YES;
  GNUNET_CONTAINER_bloomfilter_add (gi->bf,
                                    &mhash);
  return GNUNET_NO;
}

GNUNET_BLOCK_mingle_hash()

void GNUNET_BLOCK_mingle_hash	(	const struct GNUNET_HashCode *	in,
		uint32_t	mingle_number,
		struct GNUNET_HashCode *	hc
	)

Mingle hash with the mingle_number to produce different bits.

Parameters

in	original hash code
mingle_number	number for hash permutation
hc	where to store the result.

Definition at line 96 of file block.c.

{
  struct MinglePacker mp = {
    .in = *in,
    .mingle = mingle_number
  };
 
  GNUNET_CRYPTO_hash (&mp,
                      sizeof(mp),
                      hc);
}

References GNUNET_CRYPTO_hash(), and MinglePacker::in.

Referenced by bf_group_mark_seen_cb(), filtered_map_initialization(), iterator_bf_create(), and iterator_bf_reduce().

Here is the call graph for this function:

Here is the caller graph for this function:

GNUNET_BLOCK_context_create()

struct GNUNET_BLOCK_Context* GNUNET_BLOCK_context_create

(

const struct GNUNET_CONFIGURATION_Handle *

cfg

)

Create a block context.

Loads the block plugins.

Parameters

cfg	configuration to use

Returns

NULL on error

Definition at line 140 of file block.c.

{
  struct GNUNET_BLOCK_Context *ctx;
 
  ctx = GNUNET_new (struct GNUNET_BLOCK_Context);
  ctx->cfg = cfg;
  GNUNET_PLUGIN_load_all_in_context (GNUNET_OS_project_data_default (),
                                     "libgnunet_plugin_block_",
                                     (void *) cfg,
                                     &add_plugin,
                                     ctx);
  return ctx;
}

References add_plugin(), cfg, ctx, GNUNET_new, GNUNET_OS_project_data_default(), and GNUNET_PLUGIN_load_all_in_context().

Referenced by run().

Here is the call graph for this function:

Here is the caller graph for this function:

GNUNET_BLOCK_context_destroy()

void GNUNET_BLOCK_context_destroy

(

struct GNUNET_BLOCK_Context *

ctx

)

Destroy the block context.

Parameters

ctx	context to destroy

Definition at line 156 of file block.c.

{
  struct Plugin *plugin;
 
  for (unsigned int i = 0; i < ctx->num_plugins; i++)
  {
    plugin = ctx->plugins[i];
    GNUNET_break (NULL ==
                  GNUNET_PLUGIN_unload (plugin->library_name,
                                        plugin->api));
    GNUNET_free (plugin->library_name);
    GNUNET_free (plugin);
  }
  GNUNET_free (ctx->plugins);
  GNUNET_free (ctx);
}

References TestcasePlugin::api, ctx, GNUNET_break, GNUNET_free, GNUNET_PLUGIN_unload(), TestcasePlugin::library_name, and plugin.

Referenced by libgnunet_plugin_block_consensus_done(), and shutdown_task().

Here is the call graph for this function:

Here is the caller graph for this function:

GNUNET_BLOCK_group_create()

struct GNUNET_BLOCK_Group* GNUNET_BLOCK_group_create	(	struct GNUNET_BLOCK_Context *	ctx,
		enum GNUNET_BLOCK_Type	type,
		const void *	raw_data,
		size_t	raw_data_size,
			...
	)

Create a new block group.

Parameters

ctx	block context in which the block group is created
type	type of the block for which we are creating the group
raw_data	optional serialized prior state of the group, NULL if unavailable/fresh
raw_data_size	number of bytes in raw_data, 0 if unavailable/fresh
...	type-specific additional data, can be empty

Returns

block group handle, NULL if block groups are not supported by this type of block (this is not an error)

Definition at line 247 of file block.c.

{
  struct GNUNET_BLOCK_PluginFunctions *plugin;
  struct GNUNET_BLOCK_Group *bg;
  va_list ap;
 
  plugin = find_plugin (ctx,
                        type);
  if (NULL == plugin)
    return NULL;
  if (NULL == plugin->create_group)
    return NULL;
  va_start (ap,
            raw_data_size);
  bg = plugin->create_group (plugin->cls,
                             type,
                             raw_data,
                             raw_data_size,
                             ap);
  va_end (ap);
  return bg;
}

Referenced by GSF_pending_request_create_(), handle_dht_p2p_get(), refresh_bloomfilter(), send_find_peer_message(), and transmit_request().

Here is the caller graph for this function:

GNUNET_BLOCK_group_serialize()

enum GNUNET_GenericReturnValue GNUNET_BLOCK_group_serialize	(	struct GNUNET_BLOCK_Group *	bg,
		void **	raw_data,
		size_t *	raw_data_size
	)

Serialize state of a block group.

Parameters

	bg	group to serialize
[out]	raw_data	set to the serialized state
[out]	raw_data_size	set to the number of bytes in raw_data

Returns

GNUNET_OK on success, GNUNET_NO if serialization is not supported, GNUNET_SYSERR on error

Definition at line 156 of file block.c.

{
  *raw_data = NULL;
  *raw_data_size = 0;
  if (NULL == bg)
    return GNUNET_NO;
  if (NULL == bg->serialize_cb)
    return GNUNET_NO;
  return bg->serialize_cb (bg,
                           raw_data,
                           raw_data_size);
}

Referenced by GSF_pending_request_get_message_().

Here is the caller graph for this function:

GNUNET_BLOCK_group_destroy()

void GNUNET_BLOCK_group_destroy

(

struct GNUNET_BLOCK_Group *

bg

)

Destroy resources used by a block group.

Parameters

bg	group to destroy, NULL is allowed

Definition at line 192 of file block.c.

{
  if (NULL == bg)
    return;
  bg->destroy_cb (bg);
}

References GNUNET_BLOCK_Group::destroy_cb.

Referenced by clean_request(), expire_oldest_entry(), refresh_bloomfilter(), send_find_peer_message(), and transmit_request().

Here is the caller graph for this function:

GNUNET_BLOCK_check_reply()

enum GNUNET_BLOCK_ReplyEvaluationResult GNUNET_BLOCK_check_reply	(	struct GNUNET_BLOCK_Context *	ctx,
		enum GNUNET_BLOCK_Type	type,
		struct GNUNET_BLOCK_Group *	group,
		const struct GNUNET_HashCode *	query,
		const void *	xquery,
		size_t	xquery_size,
		const void *	reply_block,
		size_t	reply_block_size
	)

Function called to validate if a reply is good for a particular query.

Parameters

	ctx	block contxt
	type	block type
[in,out]	group	block group to use for evaluation
	query	original query (hash)
	xquery	extrended query data (can be NULL, depending on type)
	xquery_size	number of bytes in xquery
	reply_block	response to validate
	reply_block_size	number of bytes in reply_block

Returns

characterization of result

Definition at line 247 of file block.c.

{
  struct GNUNET_BLOCK_PluginFunctions *plugin = find_plugin (ctx,
                                                             type);
 
  if (NULL == plugin)
    return GNUNET_BLOCK_REPLY_TYPE_NOT_SUPPORTED;
  return plugin->check_reply (plugin->cls,
                              type,
                              group,
                              query,
                              xquery,
                              xquery_size,
                              reply_block,
                              reply_block_size);
}

Referenced by handle_find_local_hello(), and handle_find_my_hello().

Here is the caller graph for this function:

GNUNET_BLOCK_check_query()

enum GNUNET_GenericReturnValue GNUNET_BLOCK_check_query	(	struct GNUNET_BLOCK_Context *	ctx,
		enum GNUNET_BLOCK_Type	type,
		const struct GNUNET_HashCode *	query,
		const void *	xquery,
		size_t	xquery_size
	)

Function called to validate a request.

Parameters

ctx	block contxt
type	block type
query	original query (hash)
xquery	extrended query data (can be NULL, depending on type)
xquery_size	number of bytes in xquery

Returns

GNUNET_OK if the block is fine, GNUNET_NO if not, GNUNET_SYSERR if type is not supported

Definition at line 247 of file block.c.

{
  struct GNUNET_BLOCK_PluginFunctions *plugin;
 
  if (GNUNET_BLOCK_TYPE_ANY == type)
    return GNUNET_SYSERR; /* no checks */
  plugin = find_plugin (ctx,
                        type);
  if (NULL == plugin)
    return GNUNET_SYSERR;
  return plugin->check_query (plugin->cls,
                              type,
                              query,
                              xquery,
                              xquery_size);
}

Referenced by handle_dht_p2p_get().

Here is the caller graph for this function:

GNUNET_BLOCK_check_block()

enum GNUNET_GenericReturnValue GNUNET_BLOCK_check_block	(	struct GNUNET_BLOCK_Context *	ctx,
		enum GNUNET_BLOCK_Type	type,
		const void *	block,
		size_t	block_size
	)

Function called to validate a block.

Parameters

ctx	block contxt
type	block type
block	payload to put
block_size	number of bytes in block

Returns

GNUNET_OK if the block is fine, GNUNET_NO if not, GNUNET_SYSERR if type is not supported

Definition at line 247 of file block.c.

{
  struct GNUNET_BLOCK_PluginFunctions *plugin = find_plugin (ctx,
                                                             type);
 
  if (NULL == plugin)
    return GNUNET_SYSERR;
  return plugin->check_block (plugin->cls,
                              type,
                              block,
                              block_size);
}

References ctx, find_plugin(), plugin, and type.

Referenced by cadet_reply_proc(), check_dht_p2p_hello(), handle_dht_local_put(), handle_dht_p2p_put(), and handle_p2p_put().

Here is the call graph for this function:

Here is the caller graph for this function:

GNUNET_BLOCK_get_key()

enum GNUNET_GenericReturnValue GNUNET_BLOCK_get_key	(	struct GNUNET_BLOCK_Context *	ctx,
		enum GNUNET_BLOCK_Type	type,
		const void *	block,
		size_t	block_size,
		struct GNUNET_HashCode *	key
	)

Function called to obtain the key for a block.

If the block is malformed, the function should zero-out key and return GNUNET_OK.

Parameters

ctx	block context
type	block type
block	block to get the key for
block_size	number of bytes in block
key	set to the key (query) for the given block

Returns

GNUNET_YES on success, GNUNET_NO if extracting a key from a block of this type does not work GNUNET_SYSERR if type not supported

Definition at line 247 of file block.c.

{
  struct GNUNET_BLOCK_PluginFunctions *plugin = find_plugin (ctx,
                                                             type);
 
  if (NULL == plugin)
    return GNUNET_SYSERR;
  return plugin->get_key (plugin->cls,
                          type,
                          block,
                          block_size,
                          key);
}

Referenced by cadet_reply_proc(), check_dht_p2p_hello(), handle_dht_p2p_put(), handle_p2p_put(), handle_reply(), and process_local_reply().

Here is the caller graph for this function:

GNUNET_BLOCK_group_set_seen()

enum GNUNET_GenericReturnValue GNUNET_BLOCK_group_set_seen	(	struct GNUNET_BLOCK_Group *	bg,
		const struct GNUNET_HashCode *	seen_results,
		unsigned int	seen_results_count
	)

Update block group to filter out the given results.

Note that the use of a hash for seen results implies that the caller magically knows how the specific block engine hashes for filtering duplicates, so this API may not always apply.

Parameters

bf_mutator	mutation value to use
seen_results	results already seen
seen_results_count	number of entries in seen_results

Returns

GNUNET_SYSERR if not supported, GNUNET_OK on success

Definition at line 247 of file block.c.

{
  if (NULL == bg)
    return GNUNET_OK;
  if (NULL == bg->mark_seen_cb)
    return GNUNET_SYSERR;
  bg->mark_seen_cb (bg,
                    seen_results,
                    seen_results_count);
  return GNUNET_OK;
}

Referenced by GSF_pending_request_update_(), refresh_bloomfilter(), and transmit_request().

Here is the caller graph for this function:

GNUNET_BLOCK_group_merge()

enum GNUNET_GenericReturnValue GNUNET_BLOCK_group_merge	(	struct GNUNET_BLOCK_Group *	bg1,
		struct GNUNET_BLOCK_Group *	bg2
	)

Try merging two block groups.

Afterwards, bg1 should remain valid and contain the rules from both bg1 and bg2, and bg2 should be destroyed (as part of this call). The latter should happen even if merging is not supported.

Parameters

[in,out]	bg1	first group to merge, is updated
	bg2	second group to merge, is destroyed

Returns

GNUNET_OK on success, GNUNET_NO if merge failed due to different nonce GNUNET_SYSERR if merging is not supported

Definition at line 192 of file block.c.

{
  enum GNUNET_GenericReturnValue ret;
 
  if (NULL == bg2)
    return GNUNET_OK;
  if (NULL == bg1)
  {
    bg2->destroy_cb (bg2);
    return GNUNET_OK;
  }
  if (NULL == bg1->merge_cb)
    return GNUNET_SYSERR;
  GNUNET_assert (bg1->merge_cb == bg1->merge_cb);
  ret = bg1->merge_cb (bg1,
                       bg2);
  bg2->destroy_cb (bg2);
  return ret;
}