GNUnet  0.11.x
Data Structures | Functions | Variables
gnunet-chk Namespace Reference

Data Structures

class  AESKey
 
class  Chk
 

Functions

def encode_data_to_string (data)
 
def sha512_hash (data)
 
def setup_aes_cipher_ (aes_key)
 
def aes_pad_ (data)
 
def aes_encrypt (aes_key, data)
 
def aes_decrypt (aes_key, data)
 
def compute_depth_ (size)
 
def compute_tree_size_ (depth)
 
def compute_chk_offset_ (depth, end_offset)
 
def compute_iblock_size_ (depth, offset)
 
def compute_rootchk (readin, size)
 
def chkuri_from_path (path)
 
def usage ()
 

Variables

tuple DBLOCK_SIZE = (32 * 1024)
 
int CHK_PER_INODE = 256
 
int CHK_HASH_SIZE = 64
 
int CHK_QUERY_SIZE = CHK_HASH_SIZE
 
string GNUNET_FS_URI_PREFIX = "gnunet://fs/"
 
string GNUNET_FS_URI_CHK_INFIX = "chk/"
 
 opts
 
 args
 

Function Documentation

◆ encode_data_to_string()

def gnunet-chk.encode_data_to_string (   data)
Returns an ASCII encoding of the given data block like
GNUNET_STRINGS_data_to_string() function.

data: A bytearray representing the block of data which has to be encoded

Definition at line 51 of file gnunet-chk.py.

References len, and type.

Referenced by gnunet-chk.Chk.uri().

51 def encode_data_to_string(data):
52  """Returns an ASCII encoding of the given data block like
53  GNUNET_STRINGS_data_to_string() function.
54 
55  data: A bytearray representing the block of data which has to be encoded
56  """
57  echart = "0123456789ABCDEFGHIJKLMNOPQRSTUV"
58  assert (None != data)
59  assert (bytearray == type(data))
60  size = len(data)
61  assert (0 != size)
62  vbit = 0
63  wpos = 0
64  rpos = 0
65  bits = 0
66  out = ""
67  while (rpos < size) or (vbit > 0):
68  if (rpos < size) and (vbit < 5):
69  bits = (bits << 8) | data[rpos] # eat 8 more bits
70  rpos += 1
71  vbit += 8
72  if (vbit < 5):
73  bits <<= (5 - vbit) # zero-padding
74  assert (vbit == ((size * 8) % 5))
75  vbit = 5
76  out += echart[(bits >> (vbit - 5)) & 31]
77  wpos += 1
78  vbit -= 5
79  assert (0 == vbit)
80  return out
81 
82 
def encode_data_to_string(data)
Definition: gnunet-chk.py:51
enum GNUNET_TESTBED_UnderlayLinkModelType type
the type of this model
uint16_t len
length of data (which is always a uint32_t, but presumably this can be used to specify that fewer byt...
Here is the caller graph for this function:

◆ sha512_hash()

def gnunet-chk.sha512_hash (   data)
Returns the sha512 hash of the given data.

data: string to hash

Definition at line 83 of file gnunet-chk.py.

Referenced by compute_rootchk().

83 def sha512_hash(data):
84  """ Returns the sha512 hash of the given data.
85 
86  data: string to hash
87  """
88  hash_obj = sha512()
89  hash_obj.update(data)
90  return hash_obj.digest()
91 
92 
def sha512_hash(data)
Definition: gnunet-chk.py:83
Here is the caller graph for this function:

◆ setup_aes_cipher_()

def gnunet-chk.setup_aes_cipher_ (   aes_key)
Initializes the AES object with settings similar to those in GNUnet.

aes_key: the AESKey object
Returns the newly initialized AES object

Definition at line 127 of file gnunet-chk.py.

Referenced by aes_decrypt(), and aes_encrypt().

127 def setup_aes_cipher_(aes_key):
128  """Initializes the AES object with settings similar to those in GNUnet.
129 
130  aes_key: the AESKey object
131  Returns the newly initialized AES object
132  """
133  return AES.new(aes_key.key, AES.MODE_CFB, aes_key.iv, segment_size=128)
134 
135 
def setup_aes_cipher_(aes_key)
Definition: gnunet-chk.py:127
Here is the caller graph for this function:

◆ aes_pad_()

def gnunet-chk.aes_pad_ (   data)
Adds padding to the data such that the size of the data is a multiple of
16 bytes

data: the data string
Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
as padding; data is the new data string with padded bytes at the end

Definition at line 136 of file gnunet-chk.py.

References len.

Referenced by aes_decrypt(), and aes_encrypt().

136 def aes_pad_(data):
137  """Adds padding to the data such that the size of the data is a multiple of
138  16 bytes
139 
140  data: the data string
141  Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
142  as padding; data is the new data string with padded bytes at the end
143  """
144  pad_len = len(data) % 16
145  if (0 != pad_len):
146  pad_len = 16 - pad_len
147  pad_bytes = bytearray(15)
148  data += str(pad_bytes[:pad_len])
149  return (pad_len, data)
150 
151 
def aes_pad_(data)
Definition: gnunet-chk.py:136
uint16_t len
length of data (which is always a uint32_t, but presumably this can be used to specify that fewer byt...
Here is the caller graph for this function:

◆ aes_encrypt()

def gnunet-chk.aes_encrypt (   aes_key,
  data 
)
Encrypts the given data using AES.

aes_key: the AESKey object to use for AES encryption
data: the data string to encrypt

Definition at line 152 of file gnunet-chk.py.

References aes_pad_(), and setup_aes_cipher_().

Referenced by compute_rootchk().

152 def aes_encrypt(aes_key, data):
153  """Encrypts the given data using AES.
154 
155  aes_key: the AESKey object to use for AES encryption
156  data: the data string to encrypt
157  """
158  (pad_len, data) = aes_pad_(data)
159  cipher = setup_aes_cipher_(aes_key)
160  enc_data = cipher.encrypt(data)
161  if (0 != pad_len):
162  enc_data = enc_data[:-pad_len]
163  return enc_data
164 
165 
def setup_aes_cipher_(aes_key)
Definition: gnunet-chk.py:127
def aes_pad_(data)
Definition: gnunet-chk.py:136
def aes_encrypt(aes_key, data)
Definition: gnunet-chk.py:152
Here is the call graph for this function:
Here is the caller graph for this function:

◆ aes_decrypt()

def gnunet-chk.aes_decrypt (   aes_key,
  data 
)
Decrypts the given data using AES

aes_key: the AESKey object to use for AES decryption
data: the data string to decrypt

Definition at line 166 of file gnunet-chk.py.

References aes_pad_(), and setup_aes_cipher_().

166 def aes_decrypt(aes_key, data):
167  """Decrypts the given data using AES
168 
169  aes_key: the AESKey object to use for AES decryption
170  data: the data string to decrypt
171  """
172  (pad_len, data) = aes_pad_(data)
173  cipher = setup_aes_cipher_(aes_key)
174  ptext = cipher.decrypt(data)
175  if (0 != pad_len):
176  ptext = ptext[:-pad_len]
177  return ptext
178 
179 
def setup_aes_cipher_(aes_key)
Definition: gnunet-chk.py:127
def aes_pad_(data)
Definition: gnunet-chk.py:136
def aes_decrypt(aes_key, data)
Definition: gnunet-chk.py:166
Here is the call graph for this function:

◆ compute_depth_()

def gnunet-chk.compute_depth_ (   size)
Computes the depth of the hash tree.

size: the size of the file whose tree's depth has to be computed
Returns the depth of the tree. Always > 0.

Definition at line 205 of file gnunet-chk.py.

Referenced by compute_rootchk().

205 def compute_depth_(size):
206  """Computes the depth of the hash tree.
207 
208  size: the size of the file whose tree's depth has to be computed
209  Returns the depth of the tree. Always > 0.
210  """
211  depth = 1
212  fl = DBLOCK_SIZE
213  while (fl < size):
214  depth += 1
215  if ((fl * CHK_PER_INODE) < fl):
216  return depth
217  fl = fl * CHK_PER_INODE
218  return depth
219 
220 
def compute_depth_(size)
Definition: gnunet-chk.py:205
Here is the caller graph for this function:

◆ compute_tree_size_()

def gnunet-chk.compute_tree_size_ (   depth)
Calculate how many bytes of payload a block tree of the given depth MAY
 correspond to at most (this function ignores the fact that some blocks will
 only be present partially due to the total file size cutting some blocks
 off at the end).

 depth: depth of the block.  depth==0 is a DBLOCK.
 Returns the number of bytes of payload a subtree of this depth may
 correspond to.

Definition at line 221 of file gnunet-chk.py.

Referenced by compute_chk_offset_(), and compute_iblock_size_().

221 def compute_tree_size_(depth):
222  """Calculate how many bytes of payload a block tree of the given depth MAY
223  correspond to at most (this function ignores the fact that some blocks will
224  only be present partially due to the total file size cutting some blocks
225  off at the end).
226 
227  depth: depth of the block. depth==0 is a DBLOCK.
228  Returns the number of bytes of payload a subtree of this depth may
229  correspond to.
230  """
231  rsize = DBLOCK_SIZE
232  for cnt in range(0, depth):
233  rsize *= CHK_PER_INODE
234  return rsize
235 
236 
def compute_tree_size_(depth)
Definition: gnunet-chk.py:221
Here is the caller graph for this function:

◆ compute_chk_offset_()

def gnunet-chk.compute_chk_offset_ (   depth,
  end_offset 
)
Compute the offset of the CHK for the current block in the IBlock
above

depth: depth of the IBlock in the tree (aka overall number of tree levels
         minus depth); 0 == DBLOCK
end_offset: current offset in the overall file, at the *beginning* of the
              block for DBLOCK (depth == 0), otherwise at the *end* of the
              block (exclusive)
Returns the offset in the list of CHKs in the above IBlock

Definition at line 237 of file gnunet-chk.py.

References compute_tree_size_().

Referenced by compute_rootchk().

237 def compute_chk_offset_(depth, end_offset):
238  """Compute the offset of the CHK for the current block in the IBlock
239  above
240 
241  depth: depth of the IBlock in the tree (aka overall number of tree levels
242  minus depth); 0 == DBLOCK
243  end_offset: current offset in the overall file, at the *beginning* of the
244  block for DBLOCK (depth == 0), otherwise at the *end* of the
245  block (exclusive)
246  Returns the offset in the list of CHKs in the above IBlock
247  """
248  bds = compute_tree_size_(depth)
249  if (depth > 0):
250  end_offset -= 1
251  ret = end_offset // bds
252  return ret % CHK_PER_INODE
253 
254 
def compute_chk_offset_(depth, end_offset)
Definition: gnunet-chk.py:237
def compute_tree_size_(depth)
Definition: gnunet-chk.py:221
Here is the call graph for this function:
Here is the caller graph for this function:

◆ compute_iblock_size_()

def gnunet-chk.compute_iblock_size_ (   depth,
  offset 
)
Compute the size of the current IBLOCK.  The encoder is triggering the
calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
construction.  The IBLOCK maybe a full or a partial IBLOCK, and this
function is to calculate how long it should be.

depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
         (this function is for IBLOCKs only!)
offset: current offset in the payload (!) of the overall file, must be > 0
          (since this function is called at the end of a block).
Returns the number of elements to be in the corresponding IBlock

Definition at line 255 of file gnunet-chk.py.

References compute_tree_size_().

Referenced by compute_rootchk().

255 def compute_iblock_size_(depth, offset):
256  """Compute the size of the current IBLOCK. The encoder is triggering the
257  calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
258  construction. The IBLOCK maybe a full or a partial IBLOCK, and this
259  function is to calculate how long it should be.
260 
261  depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
262  (this function is for IBLOCKs only!)
263  offset: current offset in the payload (!) of the overall file, must be > 0
264  (since this function is called at the end of a block).
265  Returns the number of elements to be in the corresponding IBlock
266  """
267  assert (depth > 0)
268  assert (offset > 0)
269  bds = compute_tree_size_(depth)
270  mod = offset % bds
271  if mod is 0:
272  ret = CHK_PER_INODE
273  else:
274  bds /= CHK_PER_INODE
275  ret = mod // bds
276  if (mod % bds) is not 0:
277  ret += 1
278  return ret
279 
280 
def compute_iblock_size_(depth, offset)
Definition: gnunet-chk.py:255
def compute_tree_size_(depth)
Definition: gnunet-chk.py:221
Here is the call graph for this function:
Here is the caller graph for this function:

◆ compute_rootchk()

def gnunet-chk.compute_rootchk (   readin,
  size 
)
Returns the content hash key after generating the hash tree for the given
input stream.

readin: the stream where to read data from
size: the size of data to be read

Definition at line 281 of file gnunet-chk.py.

References aes_encrypt(), compute_chk_offset_(), compute_depth_(), compute_iblock_size_(), len, reduce(), and sha512_hash().

Referenced by chkuri_from_path().

281 def compute_rootchk(readin, size):
282  """Returns the content hash key after generating the hash tree for the given
283  input stream.
284 
285  readin: the stream where to read data from
286  size: the size of data to be read
287  """
288  depth = compute_depth_(size)
289  current_depth = 0
290  chks = [None] * (depth * CHK_PER_INODE) # list buffer
291  read_offset = 0
292  logging.debug("Begining to calculate tree hash with depth: " + repr(depth))
293  while True:
294  if (depth == current_depth):
295  off = CHK_PER_INODE * (depth - 1)
296  assert (chks[off] is not None)
297  logging.debug("Encoding done, reading CHK `" + chks[off].query + \
298  "' from " + repr(off) + "\n")
299  uri_chk = chks[off]
300  assert (size == read_offset)
301  uri_chk.setSize(size)
302  return uri_chk
303  if (0 == current_depth):
304  pt_size = min(DBLOCK_SIZE, size - read_offset)
305  try:
306  pt_block = readin.read(pt_size)
307  except IOError:
308  logging.warning("Error reading input file stream")
309  return None
310  else:
311  pt_elements = compute_iblock_size_(current_depth, read_offset)
312  pt_block = ""
313  pt_block = \
314  reduce((lambda ba, chk:
315  ba + (chk.key + chk.query)),
316  chks[(current_depth - 1) * CHK_PER_INODE:][:pt_elements],
317  pt_block)
318  pt_size = pt_elements * (CHK_HASH_SIZE + CHK_QUERY_SIZE)
319  assert (len(pt_block) == pt_size)
320  assert (pt_size <= DBLOCK_SIZE)
321  off = compute_chk_offset_(current_depth, read_offset)
322  logging.debug("Encoding data at offset " + repr(read_offset) + \
323  " and depth " + repr(current_depth) + " with block " \
324  "size " + repr(pt_size) + " and target CHK offset " + \
325  repr(current_depth * CHK_PER_INODE))
326  pt_hash = sha512_hash(pt_block)
327  pt_aes_key = AESKey(pt_hash)
328  pt_enc = aes_encrypt(pt_aes_key, pt_block)
329  pt_enc_hash = sha512_hash(pt_enc)
330  chk = Chk(pt_hash, pt_enc_hash)
331  chks[(current_depth * CHK_PER_INODE) + off] = chk
332  if (0 == current_depth):
333  read_offset += pt_size
334  if (read_offset == size) or \
335  (0 == (read_offset % (CHK_PER_INODE * DBLOCK_SIZE))):
336  current_depth += 1
337  else:
338  if (CHK_PER_INODE == off) or (read_offset == size):
339  current_depth += 1
340  else:
341  current_depth = 0
342 
343 
def compute_iblock_size_(depth, offset)
Definition: gnunet-chk.py:255
def compute_depth_(size)
Definition: gnunet-chk.py:205
def sha512_hash(data)
Definition: gnunet-chk.py:83
def compute_chk_offset_(depth, end_offset)
Definition: gnunet-chk.py:237
static void reduce(u8 *r)
def aes_encrypt(aes_key, data)
Definition: gnunet-chk.py:152
uint16_t len
length of data (which is always a uint32_t, but presumably this can be used to specify that fewer byt...
def compute_rootchk(readin, size)
Definition: gnunet-chk.py:281
Here is the call graph for this function:
Here is the caller graph for this function:

◆ chkuri_from_path()

def gnunet-chk.chkuri_from_path (   path)
Returns the CHK URI of the file at the given path.

path: the path of the file whose CHK has to be calculated

Definition at line 344 of file gnunet-chk.py.

References compute_rootchk().

344 def chkuri_from_path(path):
345  """Returns the CHK URI of the file at the given path.
346 
347  path: the path of the file whose CHK has to be calculated
348  """
349  size = os.path.getsize(path)
350  readin = open(path, "rb")
351  chk = compute_rootchk(readin, size)
352  readin.close()
353  return chk.uri()
354 
355 
def chkuri_from_path(path)
Definition: gnunet-chk.py:344
def compute_rootchk(readin, size)
Definition: gnunet-chk.py:281
Here is the call graph for this function:

◆ usage()

def gnunet-chk.usage ( )
Prints help about using this script.

Definition at line 356 of file gnunet-chk.py.

Referenced by dns_string_to_value().

356 def usage():
357  """Prints help about using this script."""
358  print("""
359 Usage: gnunet-chk.py [options] file
360 Prints the Content Hash Key of given file in GNUNET-style URI.
361 
362 Options:
363  -h, --help : prints this message
364 """)
365 
366 
def usage()
Definition: gnunet-chk.py:356
Here is the caller graph for this function:

Variable Documentation

◆ DBLOCK_SIZE

tuple gnunet-chk.DBLOCK_SIZE = (32 * 1024)

Definition at line 34 of file gnunet-chk.py.

◆ CHK_PER_INODE

int gnunet-chk.CHK_PER_INODE = 256

Definition at line 40 of file gnunet-chk.py.

◆ CHK_HASH_SIZE

int gnunet-chk.CHK_HASH_SIZE = 64

Definition at line 42 of file gnunet-chk.py.

◆ CHK_QUERY_SIZE

int gnunet-chk.CHK_QUERY_SIZE = CHK_HASH_SIZE

Definition at line 44 of file gnunet-chk.py.

◆ GNUNET_FS_URI_PREFIX

string gnunet-chk.GNUNET_FS_URI_PREFIX = "gnunet://fs/"

Definition at line 46 of file gnunet-chk.py.

◆ GNUNET_FS_URI_CHK_INFIX

string gnunet-chk.GNUNET_FS_URI_CHK_INFIX = "chk/"

Definition at line 48 of file gnunet-chk.py.

◆ opts

gnunet-chk.opts

Definition at line 369 of file gnunet-chk.py.

◆ args

gnunet-chk.args