GNUnet  0.11.x
Data Structures | Functions | Variables
gnunet-chk Namespace Reference

Data Structures

class  AESKey
 
class  Chk
 

Functions

def encode_data_to_string (data)
 
def sha512_hash (data)
 
def setup_aes_cipher_ (aes_key)
 
def aes_pad_ (data)
 
def aes_encrypt (aes_key, data)
 
def aes_decrypt (aes_key, data)
 
def compute_depth_ (size)
 
def compute_tree_size_ (depth)
 
def compute_chk_offset_ (depth, end_offset)
 
def compute_iblock_size_ (depth, offset)
 
def compute_rootchk (readin, size)
 
def chkuri_from_path (path)
 
def usage ()
 

Variables

tuple DBLOCK_SIZE = (32 * 1024)
 
int CHK_PER_INODE = 256
 
int CHK_HASH_SIZE = 64
 
int CHK_QUERY_SIZE = CHK_HASH_SIZE
 
string GNUNET_FS_URI_PREFIX = "gnunet://fs/"
 
string GNUNET_FS_URI_CHK_INFIX = "chk/"
 
 opts
 
 args
 

Function Documentation

◆ encode_data_to_string()

def gnunet-chk.encode_data_to_string (   data)
Returns an ASCII encoding of the given data block like
GNUNET_STRINGS_data_to_string() function.

data: A bytearray representing the block of data which has to be encoded

Definition at line 50 of file gnunet-chk.py.

References len, and type.

Referenced by gnunet-chk.Chk.uri().

50 def encode_data_to_string(data):
51  """Returns an ASCII encoding of the given data block like
52  GNUNET_STRINGS_data_to_string() function.
53 
54  data: A bytearray representing the block of data which has to be encoded
55  """
56  echart = "0123456789ABCDEFGHIJKLMNOPQRSTUV"
57  assert (None != data)
58  assert (bytearray == type(data))
59  size = len(data)
60  assert (0 != size)
61  vbit = 0
62  wpos = 0
63  rpos = 0
64  bits = 0
65  out = ""
66  while (rpos < size) or (vbit > 0):
67  if (rpos < size) and (vbit < 5):
68  bits = (bits << 8) | data[rpos] # eat 8 more bits
69  rpos += 1
70  vbit += 8
71  if (vbit < 5):
72  bits <<= (5 - vbit) # zero-padding
73  assert (vbit == ((size * 8) % 5))
74  vbit = 5
75  out += echart[(bits >> (vbit - 5)) & 31]
76  wpos += 1
77  vbit -= 5
78  assert (0 == vbit)
79  return out
80 
81 
def encode_data_to_string(data)
Definition: gnunet-chk.py:50
enum GNUNET_TESTBED_UnderlayLinkModelType type
the type of this model
uint16_t len
length of data (which is always a uint32_t, but presumably this can be used to specify that fewer byt...
Here is the caller graph for this function:

◆ sha512_hash()

def gnunet-chk.sha512_hash (   data)
Returns the sha512 hash of the given data.

data: string to hash

Definition at line 82 of file gnunet-chk.py.

Referenced by compute_rootchk().

82 def sha512_hash(data):
83  """ Returns the sha512 hash of the given data.
84 
85  data: string to hash
86  """
87  hash_obj = sha512()
88  hash_obj.update(data)
89  return hash_obj.digest()
90 
91 
def sha512_hash(data)
Definition: gnunet-chk.py:82
Here is the caller graph for this function:

◆ setup_aes_cipher_()

def gnunet-chk.setup_aes_cipher_ (   aes_key)
Initializes the AES object with settings similar to those in GNUnet.

aes_key: the AESKey object
Returns the newly initialized AES object

Definition at line 126 of file gnunet-chk.py.

Referenced by aes_decrypt(), and aes_encrypt().

126 def setup_aes_cipher_(aes_key):
127  """Initializes the AES object with settings similar to those in GNUnet.
128 
129  aes_key: the AESKey object
130  Returns the newly initialized AES object
131  """
132  return AES.new(aes_key.key, AES.MODE_CFB, aes_key.iv, segment_size=128)
133 
134 
def setup_aes_cipher_(aes_key)
Definition: gnunet-chk.py:126
Here is the caller graph for this function:

◆ aes_pad_()

def gnunet-chk.aes_pad_ (   data)
Adds padding to the data such that the size of the data is a multiple of
16 bytes

data: the data string
Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
as padding; data is the new data string with padded bytes at the end

Definition at line 135 of file gnunet-chk.py.

References len.

Referenced by aes_decrypt(), and aes_encrypt().

135 def aes_pad_(data):
136  """Adds padding to the data such that the size of the data is a multiple of
137  16 bytes
138 
139  data: the data string
140  Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
141  as padding; data is the new data string with padded bytes at the end
142  """
143  pad_len = len(data) % 16
144  if (0 != pad_len):
145  pad_len = 16 - pad_len
146  pad_bytes = bytearray(15)
147  data += str(pad_bytes[:pad_len])
148  return (pad_len, data)
149 
150 
def aes_pad_(data)
Definition: gnunet-chk.py:135
uint16_t len
length of data (which is always a uint32_t, but presumably this can be used to specify that fewer byt...
Here is the caller graph for this function:

◆ aes_encrypt()

def gnunet-chk.aes_encrypt (   aes_key,
  data 
)
Encrypts the given data using AES.

aes_key: the AESKey object to use for AES encryption
data: the data string to encrypt

Definition at line 151 of file gnunet-chk.py.

References aes_pad_(), and setup_aes_cipher_().

Referenced by compute_rootchk().

151 def aes_encrypt(aes_key, data):
152  """Encrypts the given data using AES.
153 
154  aes_key: the AESKey object to use for AES encryption
155  data: the data string to encrypt
156  """
157  (pad_len, data) = aes_pad_(data)
158  cipher = setup_aes_cipher_(aes_key)
159  enc_data = cipher.encrypt(data)
160  if (0 != pad_len):
161  enc_data = enc_data[:-pad_len]
162  return enc_data
163 
164 
def setup_aes_cipher_(aes_key)
Definition: gnunet-chk.py:126
def aes_pad_(data)
Definition: gnunet-chk.py:135
def aes_encrypt(aes_key, data)
Definition: gnunet-chk.py:151
Here is the call graph for this function:
Here is the caller graph for this function:

◆ aes_decrypt()

def gnunet-chk.aes_decrypt (   aes_key,
  data 
)
Decrypts the given data using AES

aes_key: the AESKey object to use for AES decryption
data: the data string to decrypt

Definition at line 165 of file gnunet-chk.py.

References aes_pad_(), and setup_aes_cipher_().

165 def aes_decrypt(aes_key, data):
166  """Decrypts the given data using AES
167 
168  aes_key: the AESKey object to use for AES decryption
169  data: the data string to decrypt
170  """
171  (pad_len, data) = aes_pad_(data)
172  cipher = setup_aes_cipher_(aes_key)
173  ptext = cipher.decrypt(data)
174  if (0 != pad_len):
175  ptext = ptext[:-pad_len]
176  return ptext
177 
178 
def setup_aes_cipher_(aes_key)
Definition: gnunet-chk.py:126
def aes_pad_(data)
Definition: gnunet-chk.py:135
def aes_decrypt(aes_key, data)
Definition: gnunet-chk.py:165
Here is the call graph for this function:

◆ compute_depth_()

def gnunet-chk.compute_depth_ (   size)
Computes the depth of the hash tree.

size: the size of the file whose tree's depth has to be computed
Returns the depth of the tree. Always > 0.

Definition at line 204 of file gnunet-chk.py.

Referenced by compute_rootchk().

204 def compute_depth_(size):
205  """Computes the depth of the hash tree.
206 
207  size: the size of the file whose tree's depth has to be computed
208  Returns the depth of the tree. Always > 0.
209  """
210  depth = 1
211  fl = DBLOCK_SIZE
212  while (fl < size):
213  depth += 1
214  if ((fl * CHK_PER_INODE) < fl):
215  return depth
216  fl = fl * CHK_PER_INODE
217  return depth
218 
219 
def compute_depth_(size)
Definition: gnunet-chk.py:204
Here is the caller graph for this function:

◆ compute_tree_size_()

def gnunet-chk.compute_tree_size_ (   depth)
Calculate how many bytes of payload a block tree of the given depth MAY
 correspond to at most (this function ignores the fact that some blocks will
 only be present partially due to the total file size cutting some blocks
 off at the end).

 depth: depth of the block.  depth==0 is a DBLOCK.
 Returns the number of bytes of payload a subtree of this depth may
 correspond to.

Definition at line 220 of file gnunet-chk.py.

Referenced by compute_chk_offset_(), and compute_iblock_size_().

220 def compute_tree_size_(depth):
221  """Calculate how many bytes of payload a block tree of the given depth MAY
222  correspond to at most (this function ignores the fact that some blocks will
223  only be present partially due to the total file size cutting some blocks
224  off at the end).
225 
226  depth: depth of the block. depth==0 is a DBLOCK.
227  Returns the number of bytes of payload a subtree of this depth may
228  correspond to.
229  """
230  rsize = DBLOCK_SIZE
231  for cnt in range(0, depth):
232  rsize *= CHK_PER_INODE
233  return rsize
234 
235 
def compute_tree_size_(depth)
Definition: gnunet-chk.py:220
Here is the caller graph for this function:

◆ compute_chk_offset_()

def gnunet-chk.compute_chk_offset_ (   depth,
  end_offset 
)
Compute the offset of the CHK for the current block in the IBlock
above

depth: depth of the IBlock in the tree (aka overall number of tree levels
         minus depth); 0 == DBLOCK
end_offset: current offset in the overall file, at the *beginning* of the
              block for DBLOCK (depth == 0), otherwise at the *end* of the
              block (exclusive)
Returns the offset in the list of CHKs in the above IBlock

Definition at line 236 of file gnunet-chk.py.

References compute_tree_size_().

Referenced by compute_rootchk().

236 def compute_chk_offset_(depth, end_offset):
237  """Compute the offset of the CHK for the current block in the IBlock
238  above
239 
240  depth: depth of the IBlock in the tree (aka overall number of tree levels
241  minus depth); 0 == DBLOCK
242  end_offset: current offset in the overall file, at the *beginning* of the
243  block for DBLOCK (depth == 0), otherwise at the *end* of the
244  block (exclusive)
245  Returns the offset in the list of CHKs in the above IBlock
246  """
247  bds = compute_tree_size_(depth)
248  if (depth > 0):
249  end_offset -= 1
250  ret = end_offset // bds
251  return ret % CHK_PER_INODE
252 
253 
def compute_chk_offset_(depth, end_offset)
Definition: gnunet-chk.py:236
def compute_tree_size_(depth)
Definition: gnunet-chk.py:220
Here is the call graph for this function:
Here is the caller graph for this function:

◆ compute_iblock_size_()

def gnunet-chk.compute_iblock_size_ (   depth,
  offset 
)
Compute the size of the current IBLOCK.  The encoder is triggering the
calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
construction.  The IBLOCK maybe a full or a partial IBLOCK, and this
function is to calculate how long it should be.

depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
         (this function is for IBLOCKs only!)
offset: current offset in the payload (!) of the overall file, must be > 0
          (since this function is called at the end of a block).
Returns the number of elements to be in the corresponding IBlock

Definition at line 254 of file gnunet-chk.py.

References compute_tree_size_().

Referenced by compute_rootchk().

254 def compute_iblock_size_(depth, offset):
255  """Compute the size of the current IBLOCK. The encoder is triggering the
256  calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
257  construction. The IBLOCK maybe a full or a partial IBLOCK, and this
258  function is to calculate how long it should be.
259 
260  depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
261  (this function is for IBLOCKs only!)
262  offset: current offset in the payload (!) of the overall file, must be > 0
263  (since this function is called at the end of a block).
264  Returns the number of elements to be in the corresponding IBlock
265  """
266  assert (depth > 0)
267  assert (offset > 0)
268  bds = compute_tree_size_(depth)
269  mod = offset % bds
270  if mod is 0:
271  ret = CHK_PER_INODE
272  else:
273  bds /= CHK_PER_INODE
274  ret = mod // bds
275  if (mod % bds) is not 0:
276  ret += 1
277  return ret
278 
279 
def compute_iblock_size_(depth, offset)
Definition: gnunet-chk.py:254
def compute_tree_size_(depth)
Definition: gnunet-chk.py:220
Here is the call graph for this function:
Here is the caller graph for this function:

◆ compute_rootchk()

def gnunet-chk.compute_rootchk (   readin,
  size 
)
Returns the content hash key after generating the hash tree for the given
input stream.

readin: the stream where to read data from
size: the size of data to be read

Definition at line 280 of file gnunet-chk.py.

References aes_encrypt(), compute_chk_offset_(), compute_depth_(), compute_iblock_size_(), len, min, and sha512_hash().

Referenced by chkuri_from_path().

280 def compute_rootchk(readin, size):
281  """Returns the content hash key after generating the hash tree for the given
282  input stream.
283 
284  readin: the stream where to read data from
285  size: the size of data to be read
286  """
287  depth = compute_depth_(size)
288  current_depth = 0
289  chks = [None] * (depth * CHK_PER_INODE) # list buffer
290  read_offset = 0
291  logging.debug("Beginning to calculate tree hash with depth: " + repr(depth))
292  while True:
293  if (depth == current_depth):
294  off = CHK_PER_INODE * (depth - 1)
295  assert (chks[off] is not None)
296  logging.debug("Encoding done, reading CHK `" + chks[off].query + \
297  "' from " + repr(off) + "\n")
298  uri_chk = chks[off]
299  assert (size == read_offset)
300  uri_chk.setSize(size)
301  return uri_chk
302  if (0 == current_depth):
303  pt_size = min(DBLOCK_SIZE, size - read_offset)
304  try:
305  pt_block = readin.read(pt_size)
306  except IOError:
307  logging.warning("Error reading input file stream")
308  return None
309  else:
310  pt_elements = compute_iblock_size_(current_depth, read_offset)
311  pt_block = ""
312  pt_block = \
313  reduce((lambda ba, chk:
314  ba + (chk.key + chk.query)),
315  chks[(current_depth - 1) * CHK_PER_INODE:][:pt_elements],
316  pt_block)
317  pt_size = pt_elements * (CHK_HASH_SIZE + CHK_QUERY_SIZE)
318  assert (len(pt_block) == pt_size)
319  assert (pt_size <= DBLOCK_SIZE)
320  off = compute_chk_offset_(current_depth, read_offset)
321  logging.debug("Encoding data at offset " + repr(read_offset) + \
322  " and depth " + repr(current_depth) + " with block " \
323  "size " + repr(pt_size) + " and target CHK offset " + \
324  repr(current_depth * CHK_PER_INODE))
325  pt_hash = sha512_hash(pt_block)
326  pt_aes_key = AESKey(pt_hash)
327  pt_enc = aes_encrypt(pt_aes_key, pt_block)
328  pt_enc_hash = sha512_hash(pt_enc)
329  chk = Chk(pt_hash, pt_enc_hash)
330  chks[(current_depth * CHK_PER_INODE) + off] = chk
331  if (0 == current_depth):
332  read_offset += pt_size
333  if (read_offset == size) or \
334  (0 == (read_offset % (CHK_PER_INODE * DBLOCK_SIZE))):
335  current_depth += 1
336  else:
337  if (CHK_PER_INODE == off) or (read_offset == size):
338  current_depth += 1
339  else:
340  current_depth = 0
341 
342 
def compute_iblock_size_(depth, offset)
Definition: gnunet-chk.py:254
def compute_depth_(size)
Definition: gnunet-chk.py:204
def sha512_hash(data)
Definition: gnunet-chk.py:82
def compute_chk_offset_(depth, end_offset)
Definition: gnunet-chk.py:236
def aes_encrypt(aes_key, data)
Definition: gnunet-chk.py:151
#define min(x, y)
uint16_t len
length of data (which is always a uint32_t, but presumably this can be used to specify that fewer byt...
def compute_rootchk(readin, size)
Definition: gnunet-chk.py:280
Here is the call graph for this function:
Here is the caller graph for this function:

◆ chkuri_from_path()

def gnunet-chk.chkuri_from_path (   path)
Returns the CHK URI of the file at the given path.

path: the path of the file whose CHK has to be calculated

Definition at line 343 of file gnunet-chk.py.

References compute_rootchk().

343 def chkuri_from_path(path):
344  """Returns the CHK URI of the file at the given path.
345 
346  path: the path of the file whose CHK has to be calculated
347  """
348  size = os.path.getsize(path)
349  readin = open(path, "rb")
350  chk = compute_rootchk(readin, size)
351  readin.close()
352  return chk.uri()
353 
354 
def chkuri_from_path(path)
Definition: gnunet-chk.py:343
def compute_rootchk(readin, size)
Definition: gnunet-chk.py:280
Here is the call graph for this function:

◆ usage()

def gnunet-chk.usage ( )
Prints help about using this script.

Definition at line 355 of file gnunet-chk.py.

Referenced by dns_string_to_value().

355 def usage():
356  """Prints help about using this script."""
357  print(
358  """
359 Usage: gnunet-chk.py [options] file
360 Prints the Content Hash Key of given file in GNUNET-style URI.
361 
362 Options:
363  -h, --help : prints this message
364 """
365  )
366 
367 
def usage()
Definition: gnunet-chk.py:355
Here is the caller graph for this function:

Variable Documentation

◆ DBLOCK_SIZE

tuple gnunet-chk.DBLOCK_SIZE = (32 * 1024)

Definition at line 33 of file gnunet-chk.py.

◆ CHK_PER_INODE

int gnunet-chk.CHK_PER_INODE = 256

Definition at line 39 of file gnunet-chk.py.

◆ CHK_HASH_SIZE

int gnunet-chk.CHK_HASH_SIZE = 64

Definition at line 41 of file gnunet-chk.py.

◆ CHK_QUERY_SIZE

int gnunet-chk.CHK_QUERY_SIZE = CHK_HASH_SIZE

Definition at line 43 of file gnunet-chk.py.

◆ GNUNET_FS_URI_PREFIX

string gnunet-chk.GNUNET_FS_URI_PREFIX = "gnunet://fs/"

Definition at line 45 of file gnunet-chk.py.

◆ GNUNET_FS_URI_CHK_INFIX

string gnunet-chk.GNUNET_FS_URI_CHK_INFIX = "chk/"

Definition at line 47 of file gnunet-chk.py.

◆ opts

gnunet-chk.opts

Definition at line 370 of file gnunet-chk.py.

◆ args

gnunet-chk.args

Definition at line 370 of file gnunet-chk.py.