fs_directory.c File Reference

Helper functions for building directories. More...

#include "platform.h"
#include "gnunet_fs_service.h"
#include "fs_api.h"

Include dependency graph for fs_directory.c:

Go to the source code of this file.

Data Structures
struct	GetFullDataClosure
	Closure for 'find_full_data'. More...
struct	BuilderEntry
	Entries in the directory (builder). More...
struct	GNUNET_FS_DirectoryBuilder
	Internal state of a directory builder. More...

Macros
#define	GNUNET_DIRECTORY_MAGIC   "\211GND\r\n\032\n"
	String that is used to indicate that a file is a GNUnet directory. More...

Functions
int	GNUNET_FS_meta_data_test_for_directory (const struct GNUNET_FS_MetaData *md)
	Does the meta-data claim that this is a directory? Checks if the mime-type is that of a GNUnet directory. More...
void	GNUNET_FS_meta_data_make_directory (struct GNUNET_FS_MetaData *md)
	Set the MIMETYPE information for the given metadata to "application/gnunet-directory". More...
static int	find_full_data (void *cls, const char *plugin_name, enum EXTRACTOR_MetaType type, enum EXTRACTOR_MetaFormat format, const char *data_mime_type, const char *data, size_t data_len)
	Type of a function that libextractor calls for each meta data item found. More...
int	GNUNET_FS_directory_list_contents (size_t size, const void *data, uint64_t offset, GNUNET_FS_DirectoryEntryProcessor dep, void *dep_cls)
	Iterate over all entries in a directory. More...
struct GNUNET_FS_DirectoryBuilder *	GNUNET_FS_directory_builder_create (const struct GNUNET_FS_MetaData *mdir)
	Create a directory builder. More...
void	GNUNET_FS_directory_builder_add (struct GNUNET_FS_DirectoryBuilder *bld, const struct GNUNET_FS_Uri *uri, const struct GNUNET_FS_MetaData *md, const void *data)
	Add an entry to a directory. More...
static size_t	do_align (size_t start_position, size_t end_position)
	Given the start and end position of a block of data, return the end position of that data after alignment to the DBLOCK_SIZE. More...
static void	block_align (size_t start, unsigned int count, const size_t *sizes, unsigned int *perm)
	Compute a permutation of the blocks to minimize the cost of alignment. More...
int	GNUNET_FS_directory_builder_finish (struct GNUNET_FS_DirectoryBuilder *bld, size_t *rsize, void **rdata)
	Finish building the directory. More...

Detailed Description

Helper functions for building directories.

Author

Christian Grothoff

TODO:

• modify directory builder API to support incremental generation of directories (to allow directories that would not fit into memory to be created)
• modify directory processor API to support incremental iteration over FULL directories (without missing entries) to allow access to directories that do not fit entirely into memory

Definition in file fs_directory.c.

Macro Definition Documentation

GNUNET_DIRECTORY_MAGIC

#define GNUNET_DIRECTORY_MAGIC   "\211GND\r\n\032\n"

String that is used to indicate that a file is a GNUnet directory.

Definition at line 44 of file fs_directory.c.

Function Documentation

find_full_data()

static int find_full_data ( void *  cls, const char *  plugin_name, enum EXTRACTOR_MetaType  type, enum EXTRACTOR_MetaFormat  format, const char *  data_mime_type, const char *  data, size_t  data_len  )

static

Type of a function that libextractor calls for each meta data item found.

Parameters

cls	closure (user-defined)
plugin_name	name of the plugin that produced this value; special values can be used (e.g. '<zlib>' for zlib being used in the main libextractor library and yielding meta data).
type	libextractor-type describing the meta data
format	basic format information about data
data_mime_type	mime-type of data (not of the original file); can be NULL (if mime-type is not known)
data	actual meta-data found
data_len	number of bytes in data

Returns

0 to continue extracting, 1 to abort

Definition at line 136 of file fs_directory.c.

{
  struct GetFullDataClosure *gfdc = cls;
 
  if (type == EXTRACTOR_METATYPE_GNUNET_FULL_DATA)
  {
    gfdc->size = data_len;
    if (data_len > 0)
    {
      gfdc->data = GNUNET_malloc (data_len);
      GNUNET_memcpy (gfdc->data, data, data_len);
    }
    return 1;
  }
  return 0;
}

References GetFullDataClosure::data, data, GNUNET_malloc, GNUNET_memcpy, GetFullDataClosure::size, and type.

Referenced by GNUNET_FS_directory_list_contents().

Here is the caller graph for this function:

do_align()

static size_t do_align ( size_t  start_position, size_t  end_position  )

static

Given the start and end position of a block of data, return the end position of that data after alignment to the DBLOCK_SIZE.

Definition at line 488 of file fs_directory.c.

{
  size_t align;
 
  align = (end_position / DBLOCK_SIZE) * DBLOCK_SIZE;
  if ((start_position < align) && (end_position > align))
    return align + end_position - start_position;
  return end_position;
}

References DBLOCK_SIZE.

Referenced by block_align(), GNUNET_FS_directory_builder_finish(), GNUNET_SERVER_mst_receive(), and mst_receive().

Here is the caller graph for this function:

block_align()

static void block_align ( size_t  start, unsigned int  count, const size_t *  sizes, unsigned int *  perm  )

static

Compute a permutation of the blocks to minimize the cost of alignment.

Greedy packer.

Parameters

start	starting position for the first block
count	size of the two arrays
sizes	the sizes of the individual blocks
perm	the permutation of the blocks (updated)

Definition at line 509 of file fs_directory.c.

{
  unsigned int i;
  unsigned int j;
  unsigned int tmp;
  unsigned int best;
  ssize_t badness;
  size_t cpos;
  size_t cend;
  ssize_t cbad;
  unsigned int cval;
 
  cpos = start;
  for (i = 0; i < count; i++)
  {
    start = cpos;
    badness = 0x7FFFFFFF;
    best = -1;
    for (j = i; j < count; j++)
    {
      cval = perm[j];
      cend = cpos + sizes[cval];
      if (cpos % DBLOCK_SIZE == 0)
      {
        /* prefer placing the largest blocks first */
        cbad = -(cend % DBLOCK_SIZE);
      }
      else
      {
        if (cpos / DBLOCK_SIZE == cend / DBLOCK_SIZE)
        {
          /* Data fits into the same block! Prefer small left-overs! */
          cbad = DBLOCK_SIZE - cend % DBLOCK_SIZE;
        }
        else
        {
          /* Would have to waste space to re-align, add big factor, this
           * case is a real loss (proportional to space wasted)! */
          cbad = DBLOCK_SIZE * (DBLOCK_SIZE - cpos % DBLOCK_SIZE);
        }
      }
      if (cbad < badness)
      {
        best = j;
        badness = cbad;
      }
    }
    GNUNET_assert (best != -1);
    tmp = perm[i];
    perm[i] = perm[best];
    perm[best] = tmp;
    cpos += sizes[perm[i]];
    cpos = do_align (start, cpos);
  }
}

References DBLOCK_SIZE, do_align(), GNUNET_assert, and start.

Referenced by GNUNET_FS_directory_builder_finish().

Here is the call graph for this function:

Here is the caller graph for this function: