GNUnet 0.21.0
gnunet-chk Namespace Reference

Data Structures

class  AESKey
 
class  Chk
 

Functions

def encode_data_to_string (data)
 
def sha512_hash (data)
 
def setup_aes_cipher_ (aes_key)
 
def aes_pad_ (data)
 
def aes_encrypt (aes_key, data)
 
def aes_decrypt (aes_key, data)
 
def compute_depth_ (size)
 
def compute_tree_size_ (depth)
 
def compute_chk_offset_ (depth, end_offset)
 
def compute_iblock_size_ (depth, offset)
 
def compute_rootchk (readin, size)
 
def chkuri_from_path (path)
 
def usage ()
 

Variables

tuple DBLOCK_SIZE = (32 * 1024)
 
int CHK_PER_INODE = 256
 
int CHK_HASH_SIZE = 64
 
int CHK_QUERY_SIZE = CHK_HASH_SIZE
 
string GNUNET_FS_URI_PREFIX = "gnunet://fs/"
 
string GNUNET_FS_URI_CHK_INFIX = "chk/"
 
 opts
 
 args
 

Function Documentation

◆ encode_data_to_string()

def gnunet-chk.encode_data_to_string (   data)
Returns an ASCII encoding of the given data block like
GNUNET_STRINGS_data_to_string() function.

data: A bytearray representing the block of data which has to be encoded

Definition at line 50 of file gnunet-chk.py.

50def encode_data_to_string(data):
51 """Returns an ASCII encoding of the given data block like
53
54 data: A bytearray representing the block of data which has to be encoded
55 """
56 echart = "0123456789ABCDEFGHIJKLMNOPQRSTUV"
57 assert (None != data)
58 assert (bytearray == type(data))
59 size = len(data)
60 assert (0 != size)
61 vbit = 0
62 wpos = 0
63 rpos = 0
64 bits = 0
65 out = ""
66 while (rpos < size) or (vbit > 0):
67 if (rpos < size) and (vbit < 5):
68 bits = (bits << 8) | data[rpos] # eat 8 more bits
69 rpos += 1
70 vbit += 8
71 if (vbit < 5):
72 bits <<= (5 - vbit) # zero-padding
73 assert (vbit == ((size * 8) % 5))
74 vbit = 5
75 out += echart[(bits >> (vbit - 5)) & 31]
76 wpos += 1
77 vbit -= 5
78 assert (0 == vbit)
79 return out
80
81
static uint32_t type
Type string converted to DNS type value.
char * GNUNET_STRINGS_data_to_string(const void *data, size_t size, char *out, size_t out_size)
Convert binary data to ASCII encoding using CrockfordBase32.
Definition: strings.c:709
def encode_data_to_string(data)
Definition: gnunet-chk.py:50

References GNUNET_STRINGS_data_to_string(), and type.

Referenced by gnunet-chk.Chk.uri().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ sha512_hash()

def gnunet-chk.sha512_hash (   data)
 Returns the sha512 hash of the given data.

data: string to hash

Definition at line 82 of file gnunet-chk.py.

82def sha512_hash(data):
83 """ Returns the sha512 hash of the given data.
84
85 data: string to hash
86 """
87 hash_obj = sha512()
88 hash_obj.update(data)
89 return hash_obj.digest()
90
91
def sha512_hash(data)
Definition: gnunet-chk.py:82

Referenced by compute_rootchk().

Here is the caller graph for this function:

◆ setup_aes_cipher_()

def gnunet-chk.setup_aes_cipher_ (   aes_key)
Initializes the AES object with settings similar to those in GNUnet.

aes_key: the AESKey object
Returns the newly initialized AES object

Definition at line 126 of file gnunet-chk.py.

126def setup_aes_cipher_(aes_key):
127 """Initializes the AES object with settings similar to those in GNUnet.
128
129 aes_key: the AESKey object
130 Returns the newly initialized AES object
131 """
132 return AES.new(aes_key.key, AES.MODE_CFB, aes_key.iv, segment_size=128)
133
134
def setup_aes_cipher_(aes_key)
Definition: gnunet-chk.py:126

Referenced by aes_decrypt(), and aes_encrypt().

Here is the caller graph for this function:

◆ aes_pad_()

def gnunet-chk.aes_pad_ (   data)
Adds padding to the data such that the size of the data is a multiple of
16 bytes

data: the data string
Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
as padding; data is the new data string with padded bytes at the end

Definition at line 135 of file gnunet-chk.py.

135def aes_pad_(data):
136 """Adds padding to the data such that the size of the data is a multiple of
137 16 bytes
138
139 data: the data string
140 Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
141 as padding; data is the new data string with padded bytes at the end
142 """
143 pad_len = len(data) % 16
144 if (0 != pad_len):
145 pad_len = 16 - pad_len
146 pad_bytes = bytearray(15)
147 data += str(pad_bytes[:pad_len])
148 return (pad_len, data)
149
150
def aes_pad_(data)
Definition: gnunet-chk.py:135

Referenced by aes_decrypt(), and aes_encrypt().

Here is the caller graph for this function:

◆ aes_encrypt()

def gnunet-chk.aes_encrypt (   aes_key,
  data 
)
Encrypts the given data using AES.

aes_key: the AESKey object to use for AES encryption
data: the data string to encrypt

Definition at line 151 of file gnunet-chk.py.

151def aes_encrypt(aes_key, data):
152 """Encrypts the given data using AES.
153
154 aes_key: the AESKey object to use for AES encryption
155 data: the data string to encrypt
156 """
157 (pad_len, data) = aes_pad_(data)
158 cipher = setup_aes_cipher_(aes_key)
159 enc_data = cipher.encrypt(data)
160 if (0 != pad_len):
161 enc_data = enc_data[:-pad_len]
162 return enc_data
163
164
def aes_encrypt(aes_key, data)
Definition: gnunet-chk.py:151

References aes_pad_(), and setup_aes_cipher_().

Referenced by compute_rootchk().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ aes_decrypt()

def gnunet-chk.aes_decrypt (   aes_key,
  data 
)
Decrypts the given data using AES

aes_key: the AESKey object to use for AES decryption
data: the data string to decrypt

Definition at line 165 of file gnunet-chk.py.

165def aes_decrypt(aes_key, data):
166 """Decrypts the given data using AES
167
168 aes_key: the AESKey object to use for AES decryption
169 data: the data string to decrypt
170 """
171 (pad_len, data) = aes_pad_(data)
172 cipher = setup_aes_cipher_(aes_key)
173 ptext = cipher.decrypt(data)
174 if (0 != pad_len):
175 ptext = ptext[:-pad_len]
176 return ptext
177
178
def aes_decrypt(aes_key, data)
Definition: gnunet-chk.py:165

References aes_pad_(), and setup_aes_cipher_().

Here is the call graph for this function:

◆ compute_depth_()

def gnunet-chk.compute_depth_ (   size)
Computes the depth of the hash tree.

size: the size of the file whose tree's depth has to be computed
Returns the depth of the tree. Always > 0.

Definition at line 204 of file gnunet-chk.py.

204def compute_depth_(size):
205 """Computes the depth of the hash tree.
206
207 size: the size of the file whose tree's depth has to be computed
208 Returns the depth of the tree. Always > 0.
209 """
210 depth = 1
211 fl = DBLOCK_SIZE
212 while (fl < size):
213 depth += 1
214 if ((fl * CHK_PER_INODE) < fl):
215 return depth
216 fl = fl * CHK_PER_INODE
217 return depth
218
219
def compute_depth_(size)
Definition: gnunet-chk.py:204

Referenced by compute_rootchk().

Here is the caller graph for this function:

◆ compute_tree_size_()

def gnunet-chk.compute_tree_size_ (   depth)
Calculate how many bytes of payload a block tree of the given depth MAY
 correspond to at most (this function ignores the fact that some blocks will
 only be present partially due to the total file size cutting some blocks
 off at the end).

 depth: depth of the block.  depth==0 is a DBLOCK.
 Returns the number of bytes of payload a subtree of this depth may
 correspond to.

Definition at line 220 of file gnunet-chk.py.

220def compute_tree_size_(depth):
221 """Calculate how many bytes of payload a block tree of the given depth MAY
222 correspond to at most (this function ignores the fact that some blocks will
223 only be present partially due to the total file size cutting some blocks
224 off at the end).
225
226 depth: depth of the block. depth==0 is a DBLOCK.
227 Returns the number of bytes of payload a subtree of this depth may
228 correspond to.
229 """
230 rsize = DBLOCK_SIZE
231 for cnt in range(0, depth):
232 rsize *= CHK_PER_INODE
233 return rsize
234
235
def compute_tree_size_(depth)
Definition: gnunet-chk.py:220

Referenced by compute_chk_offset_(), and compute_iblock_size_().

Here is the caller graph for this function:

◆ compute_chk_offset_()

def gnunet-chk.compute_chk_offset_ (   depth,
  end_offset 
)
Compute the offset of the CHK for the current block in the IBlock
above

depth: depth of the IBlock in the tree (aka overall number of tree levels
         minus depth); 0 == DBLOCK
end_offset: current offset in the overall file, at the *beginning* of the
              block for DBLOCK (depth == 0), otherwise at the *end* of the
              block (exclusive)
Returns the offset in the list of CHKs in the above IBlock

Definition at line 236 of file gnunet-chk.py.

236def compute_chk_offset_(depth, end_offset):
237 """Compute the offset of the CHK for the current block in the IBlock
238 above
239
240 depth: depth of the IBlock in the tree (aka overall number of tree levels
241 minus depth); 0 == DBLOCK
242 end_offset: current offset in the overall file, at the *beginning* of the
243 block for DBLOCK (depth == 0), otherwise at the *end* of the
244 block (exclusive)
245 Returns the offset in the list of CHKs in the above IBlock
246 """
247 bds = compute_tree_size_(depth)
248 if (depth > 0):
249 end_offset -= 1
250 ret = end_offset // bds
251 return ret % CHK_PER_INODE
252
253
def compute_chk_offset_(depth, end_offset)
Definition: gnunet-chk.py:236

References compute_tree_size_().

Referenced by compute_rootchk().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ compute_iblock_size_()

def gnunet-chk.compute_iblock_size_ (   depth,
  offset 
)
Compute the size of the current IBLOCK.  The encoder is triggering the
calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
construction.  The IBLOCK maybe a full or a partial IBLOCK, and this
function is to calculate how long it should be.

depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
         (this function is for IBLOCKs only!)
offset: current offset in the payload (!) of the overall file, must be > 0
          (since this function is called at the end of a block).
Returns the number of elements to be in the corresponding IBlock

Definition at line 254 of file gnunet-chk.py.

254def compute_iblock_size_(depth, offset):
255 """Compute the size of the current IBLOCK. The encoder is triggering the
256 calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
257 construction. The IBLOCK maybe a full or a partial IBLOCK, and this
258 function is to calculate how long it should be.
259
260 depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
261 (this function is for IBLOCKs only!)
262 offset: current offset in the payload (!) of the overall file, must be > 0
263 (since this function is called at the end of a block).
264 Returns the number of elements to be in the corresponding IBlock
265 """
266 assert (depth > 0)
267 assert (offset > 0)
268 bds = compute_tree_size_(depth)
269 mod = offset % bds
270 if mod is 0:
271 ret = CHK_PER_INODE
272 else:
273 bds /= CHK_PER_INODE
274 ret = mod // bds
275 if (mod % bds) is not 0:
276 ret += 1
277 return ret
278
279
def compute_iblock_size_(depth, offset)
Definition: gnunet-chk.py:254

References compute_tree_size_().

Referenced by compute_rootchk().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ compute_rootchk()

def gnunet-chk.compute_rootchk (   readin,
  size 
)
Returns the content hash key after generating the hash tree for the given
input stream.

readin: the stream where to read data from
size: the size of data to be read

Definition at line 280 of file gnunet-chk.py.

280def compute_rootchk(readin, size):
281 """Returns the content hash key after generating the hash tree for the given
282 input stream.
283
284 readin: the stream where to read data from
285 size: the size of data to be read
286 """
287 depth = compute_depth_(size)
288 current_depth = 0
289 chks = [None] * (depth * CHK_PER_INODE) # list buffer
290 read_offset = 0
291 logging.debug("Beginning to calculate tree hash with depth: " + repr(depth))
292 while True:
293 if (depth == current_depth):
294 off = CHK_PER_INODE * (depth - 1)
295 assert (chks[off] is not None)
296 logging.debug("Encoding done, reading CHK `" + chks[off].query + \
297 "' from " + repr(off) + "\n")
298 uri_chk = chks[off]
299 assert (size == read_offset)
300 uri_chk.setSize(size)
301 return uri_chk
302 if (0 == current_depth):
303 pt_size = min(DBLOCK_SIZE, size - read_offset)
304 try:
305 pt_block = readin.read(pt_size)
306 except IOError:
307 logging.warning("Error reading input file stream")
308 return None
309 else:
310 pt_elements = compute_iblock_size_(current_depth, read_offset)
311 pt_block = ""
312 pt_block = \
313 reduce((lambda ba, chk:
314 ba + (chk.key + chk.query)),
315 chks[(current_depth - 1) * CHK_PER_INODE:][:pt_elements],
316 pt_block)
317 pt_size = pt_elements * (CHK_HASH_SIZE + CHK_QUERY_SIZE)
318 assert (len(pt_block) == pt_size)
319 assert (pt_size <= DBLOCK_SIZE)
320 off = compute_chk_offset_(current_depth, read_offset)
321 logging.debug("Encoding data at offset " + repr(read_offset) + \
322 " and depth " + repr(current_depth) + " with block " \
323 "size " + repr(pt_size) + " and target CHK offset " + \
324 repr(current_depth * CHK_PER_INODE))
325 pt_hash = sha512_hash(pt_block)
326 pt_aes_key = AESKey(pt_hash)
327 pt_enc = aes_encrypt(pt_aes_key, pt_block)
328 pt_enc_hash = sha512_hash(pt_enc)
329 chk = Chk(pt_hash, pt_enc_hash)
330 chks[(current_depth * CHK_PER_INODE) + off] = chk
331 if (0 == current_depth):
332 read_offset += pt_size
333 if (read_offset == size) or \
334 (0 == (read_offset % (CHK_PER_INODE * DBLOCK_SIZE))):
335 current_depth += 1
336 else:
337 if (CHK_PER_INODE == off) or (read_offset == size):
338 current_depth += 1
339 else:
340 current_depth = 0
341
342
#define min(x, y)
def compute_rootchk(readin, size)
Definition: gnunet-chk.py:280

References aes_encrypt(), compute_chk_offset_(), compute_depth_(), compute_iblock_size_(), min, and sha512_hash().

Referenced by chkuri_from_path().

Here is the call graph for this function:
Here is the caller graph for this function:

◆ chkuri_from_path()

def gnunet-chk.chkuri_from_path (   path)
Returns the CHK URI of the file at the given path.

path: the path of the file whose CHK has to be calculated

Definition at line 343 of file gnunet-chk.py.

343def chkuri_from_path(path):
344 """Returns the CHK URI of the file at the given path.
345
346 path: the path of the file whose CHK has to be calculated
347 """
348 size = os.path.getsize(path)
349 readin = open(path, "rb")
350 chk = compute_rootchk(readin, size)
351 readin.close()
352 return chk.uri()
353
354
def chkuri_from_path(path)
Definition: gnunet-chk.py:343

References compute_rootchk().

Here is the call graph for this function:

◆ usage()

def gnunet-chk.usage ( )
Prints help about using this script.

Definition at line 355 of file gnunet-chk.py.

355def usage():
356 """Prints help about using this script."""
357 print(
358 """
359Usage: gnunet-chk.py [options] file
360Prints the Content Hash Key of given file in GNUNET-style URI.
361
362Options:
363 -h, --help : prints this message
364"""
365 )
366
367
def usage()
Definition: gnunet-chk.py:355

Referenced by dns_string_to_value().

Here is the caller graph for this function:

Variable Documentation

◆ DBLOCK_SIZE

tuple gnunet-chk.DBLOCK_SIZE = (32 * 1024)

Definition at line 33 of file gnunet-chk.py.

◆ CHK_PER_INODE

int gnunet-chk.CHK_PER_INODE = 256

Definition at line 39 of file gnunet-chk.py.

◆ CHK_HASH_SIZE

int gnunet-chk.CHK_HASH_SIZE = 64

Definition at line 41 of file gnunet-chk.py.

◆ CHK_QUERY_SIZE

int gnunet-chk.CHK_QUERY_SIZE = CHK_HASH_SIZE

Definition at line 43 of file gnunet-chk.py.

◆ GNUNET_FS_URI_PREFIX

string gnunet-chk.GNUNET_FS_URI_PREFIX = "gnunet://fs/"

Definition at line 45 of file gnunet-chk.py.

◆ GNUNET_FS_URI_CHK_INFIX

string gnunet-chk.GNUNET_FS_URI_CHK_INFIX = "chk/"

Definition at line 47 of file gnunet-chk.py.

◆ opts

gnunet-chk.opts

Definition at line 370 of file gnunet-chk.py.

Referenced by OIDC_parse_authz_code(), set_listen_cb(), and task_start_reconcile().

◆ args

gnunet-chk.args

Definition at line 370 of file gnunet-chk.py.