gnunet-chk Namespace Reference

Data Structures
class	AESKey
class	Chk

Functions
def	encode_data_to_string (data)
def	sha512_hash (data)
def	setup_aes_cipher_ (aes_key)
def	aes_pad_ (data)
def	aes_encrypt (aes_key, data)
def	aes_decrypt (aes_key, data)
def	compute_depth_ (size)
def	compute_tree_size_ (depth)
def	compute_chk_offset_ (depth, end_offset)
def	compute_iblock_size_ (depth, offset)
def	compute_rootchk (readin, size)
def	chkuri_from_path (path)
def	usage ()

Variables
tuple	DBLOCK_SIZE = (32 * 1024)
int	CHK_PER_INODE = 256
int	CHK_HASH_SIZE = 64
int	CHK_QUERY_SIZE = CHK_HASH_SIZE
string	GNUNET_FS_URI_PREFIX = "gnunet://fs/"
string	GNUNET_FS_URI_CHK_INFIX = "chk/"
	opts
	args

Function Documentation

encode_data_to_string()

def gnunet-chk.encode_data_to_string

(

data

)

Returns an ASCII encoding of the given data block like
GNUNET_STRINGS_data_to_string() function.

data: A bytearray representing the block of data which has to be encoded

Definition at line 50 of file gnunet-chk.py.

def encode_data_to_string(data):
    """Returns an ASCII encoding of the given data block like
    GNUNET_STRINGS_data_to_string() function.
 
    data: A bytearray representing the block of data which has to be encoded
    """
    echart = "0123456789ABCDEFGHIJKLMNOPQRSTUV"
    assert (None != data)
    assert (bytearray == type(data))
    size = len(data)
    assert (0 != size)
    vbit = 0
    wpos = 0
    rpos = 0
    bits = 0
    out = ""
    while (rpos < size) or (vbit > 0):
        if (rpos < size) and (vbit < 5):
            bits = (bits << 8) | data[rpos]  # eat 8 more bits
            rpos += 1
            vbit += 8
        if (vbit < 5):
            bits <<= (5 - vbit)  # zero-padding
            assert (vbit == ((size * 8) % 5))
            vbit = 5
        out += echart[(bits >> (vbit - 5)) & 31]
        wpos += 1
        vbit -= 5
    assert (0 == vbit)
    return out
 
 

References GNUNET_STRINGS_data_to_string(), and type.

Referenced by gnunet-chk.Chk.uri().

Here is the call graph for this function:

Here is the caller graph for this function:

sha512_hash()

def gnunet-chk.sha512_hash

(

data

)

 Returns the sha512 hash of the given data.

data: string to hash

Definition at line 82 of file gnunet-chk.py.

def sha512_hash(data):
    """ Returns the sha512 hash of the given data.
 
    data: string to hash
    """
    hash_obj = sha512()
    hash_obj.update(data)
    return hash_obj.digest()
 
 

Referenced by compute_rootchk().

Here is the caller graph for this function:

setup_aes_cipher_()

def gnunet-chk.setup_aes_cipher_

(

aes_key

)

Initializes the AES object with settings similar to those in GNUnet.

aes_key: the AESKey object
Returns the newly initialized AES object

Definition at line 126 of file gnunet-chk.py.

def setup_aes_cipher_(aes_key):
    """Initializes the AES object with settings similar to those in GNUnet.
 
    aes_key: the AESKey object
    Returns the newly initialized AES object
    """
    return AES.new(aes_key.key, AES.MODE_CFB, aes_key.iv, segment_size=128)
 
 

Referenced by aes_decrypt(), and aes_encrypt().

Here is the caller graph for this function:

aes_pad_()

def gnunet-chk.aes_pad_

(

data

)

Adds padding to the data such that the size of the data is a multiple of
16 bytes

data: the data string
Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
as padding; data is the new data string with padded bytes at the end

Definition at line 135 of file gnunet-chk.py.

def aes_pad_(data):
    """Adds padding to the data such that the size of the data is a multiple of
    16 bytes
 
    data: the data string
    Returns a tuple:(pad_len, data). pad_len denotes the number of bytes added
    as padding; data is the new data string with padded bytes at the end
    """
    pad_len = len(data) % 16
    if (0 != pad_len):
        pad_len = 16 - pad_len
        pad_bytes = bytearray(15)
        data += str(pad_bytes[:pad_len])
    return (pad_len, data)
 
 

Referenced by aes_decrypt(), and aes_encrypt().

Here is the caller graph for this function:

aes_encrypt()

def gnunet-chk.aes_encrypt	(		aes_key,
			data
	)

Encrypts the given data using AES.

aes_key: the AESKey object to use for AES encryption
data: the data string to encrypt

Definition at line 151 of file gnunet-chk.py.

def aes_encrypt(aes_key, data):
    """Encrypts the given data using AES.
 
    aes_key: the AESKey object to use for AES encryption
    data: the data string to encrypt
    """
    (pad_len, data) = aes_pad_(data)
    cipher = setup_aes_cipher_(aes_key)
    enc_data = cipher.encrypt(data)
    if (0 != pad_len):
        enc_data = enc_data[:-pad_len]
    return enc_data
 
 

References aes_pad_(), and setup_aes_cipher_().

Referenced by compute_rootchk().

Here is the call graph for this function:

Here is the caller graph for this function:

aes_decrypt()

def gnunet-chk.aes_decrypt	(		aes_key,
			data
	)

Decrypts the given data using AES

aes_key: the AESKey object to use for AES decryption
data: the data string to decrypt

Definition at line 165 of file gnunet-chk.py.

def aes_decrypt(aes_key, data):
    """Decrypts the given data using AES
 
    aes_key: the AESKey object to use for AES decryption
    data: the data string to decrypt
    """
    (pad_len, data) = aes_pad_(data)
    cipher = setup_aes_cipher_(aes_key)
    ptext = cipher.decrypt(data)
    if (0 != pad_len):
        ptext = ptext[:-pad_len]
    return ptext
 
 

References aes_pad_(), and setup_aes_cipher_().

Here is the call graph for this function:

compute_depth_()

def gnunet-chk.compute_depth_

(

size

)

Computes the depth of the hash tree.

size: the size of the file whose tree's depth has to be computed
Returns the depth of the tree. Always > 0.

Definition at line 204 of file gnunet-chk.py.

def compute_depth_(size):
    """Computes the depth of the hash tree.
 
    size: the size of the file whose tree's depth has to be computed
    Returns the depth of the tree. Always > 0.
    """
    depth = 1
    fl = DBLOCK_SIZE
    while (fl < size):
        depth += 1
        if ((fl * CHK_PER_INODE) < fl):
            return depth
        fl = fl * CHK_PER_INODE
    return depth
 
 

Referenced by compute_rootchk().

Here is the caller graph for this function:

compute_tree_size_()

def gnunet-chk.compute_tree_size_

(

depth

)

Calculate how many bytes of payload a block tree of the given depth MAY
 correspond to at most (this function ignores the fact that some blocks will
 only be present partially due to the total file size cutting some blocks
 off at the end).

 depth: depth of the block.  depth==0 is a DBLOCK.
 Returns the number of bytes of payload a subtree of this depth may
 correspond to.

Definition at line 220 of file gnunet-chk.py.

def compute_tree_size_(depth):
    """Calculate how many bytes of payload a block tree of the given depth MAY
     correspond to at most (this function ignores the fact that some blocks will
     only be present partially due to the total file size cutting some blocks
     off at the end).
 
     depth: depth of the block.  depth==0 is a DBLOCK.
     Returns the number of bytes of payload a subtree of this depth may
     correspond to.
     """
    rsize = DBLOCK_SIZE
    for cnt in range(0, depth):
        rsize *= CHK_PER_INODE
    return rsize
 
 

Referenced by compute_chk_offset_(), and compute_iblock_size_().

Here is the caller graph for this function:

compute_chk_offset_()

def gnunet-chk.compute_chk_offset_	(		depth,
			end_offset
	)

Compute the offset of the CHK for the current block in the IBlock
above

depth: depth of the IBlock in the tree (aka overall number of tree levels
         minus depth); 0 == DBLOCK
end_offset: current offset in the overall file, at the *beginning* of the
              block for DBLOCK (depth == 0), otherwise at the *end* of the
              block (exclusive)
Returns the offset in the list of CHKs in the above IBlock

Definition at line 236 of file gnunet-chk.py.

def compute_chk_offset_(depth, end_offset):
    """Compute the offset of the CHK for the current block in the IBlock
    above
 
    depth: depth of the IBlock in the tree (aka overall number of tree levels
             minus depth); 0 == DBLOCK
    end_offset: current offset in the overall file, at the *beginning* of the
                  block for DBLOCK (depth == 0), otherwise at the *end* of the
                  block (exclusive)
    Returns the offset in the list of CHKs in the above IBlock
    """
    bds = compute_tree_size_(depth)
    if (depth > 0):
        end_offset -= 1
    ret = end_offset // bds
    return ret % CHK_PER_INODE
 
 

References compute_tree_size_().

Referenced by compute_rootchk().

Here is the call graph for this function:

Here is the caller graph for this function:

compute_iblock_size_()

def gnunet-chk.compute_iblock_size_	(		depth,
			offset
	)

Compute the size of the current IBLOCK.  The encoder is triggering the
calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
construction.  The IBLOCK maybe a full or a partial IBLOCK, and this
function is to calculate how long it should be.

depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
         (this function is for IBLOCKs only!)
offset: current offset in the payload (!) of the overall file, must be > 0
          (since this function is called at the end of a block).
Returns the number of elements to be in the corresponding IBlock

Definition at line 254 of file gnunet-chk.py.

def compute_iblock_size_(depth, offset):
    """Compute the size of the current IBLOCK.  The encoder is triggering the
    calculation of the size of an IBLOCK at the *end* (hence end_offset) of its
    construction.  The IBLOCK maybe a full or a partial IBLOCK, and this
    function is to calculate how long it should be.
 
    depth: depth of the IBlock in the tree, 0 would be a DBLOCK, must be > 0
             (this function is for IBLOCKs only!)
    offset: current offset in the payload (!) of the overall file, must be > 0
              (since this function is called at the end of a block).
    Returns the number of elements to be in the corresponding IBlock
    """
    assert (depth > 0)
    assert (offset > 0)
    bds = compute_tree_size_(depth)
    mod = offset % bds
    if mod is 0:
        ret = CHK_PER_INODE
    else:
        bds /= CHK_PER_INODE
        ret = mod // bds
        if (mod % bds) is not 0:
            ret += 1
    return ret
 
 

References compute_tree_size_().

Referenced by compute_rootchk().

Here is the call graph for this function:

Here is the caller graph for this function:

compute_rootchk()

def gnunet-chk.compute_rootchk	(		readin,
			size
	)

Returns the content hash key after generating the hash tree for the given
input stream.

readin: the stream where to read data from
size: the size of data to be read

Definition at line 280 of file gnunet-chk.py.

def compute_rootchk(readin, size):
    """Returns the content hash key after generating the hash tree for the given
    input stream.
 
    readin: the stream where to read data from
    size: the size of data to be read
    """
    depth = compute_depth_(size)
    current_depth = 0
    chks = [None] * (depth * CHK_PER_INODE)  # list buffer
    read_offset = 0
    logging.debug("Beginning to calculate tree hash with depth: " + repr(depth))
    while True:
        if (depth == current_depth):
            off = CHK_PER_INODE * (depth - 1)
            assert (chks[off] is not None)
            logging.debug("Encoding done, reading CHK `" + chks[off].query + \
                          "' from " + repr(off) + "\n")
            uri_chk = chks[off]
            assert (size == read_offset)
            uri_chk.setSize(size)
            return uri_chk
        if (0 == current_depth):
            pt_size = min(DBLOCK_SIZE, size - read_offset)
            try:
                pt_block = readin.read(pt_size)
            except IOError:
                logging.warning("Error reading input file stream")
                return None
        else:
            pt_elements = compute_iblock_size_(current_depth, read_offset)
            pt_block = ""
            pt_block = \
                reduce((lambda ba, chk:
                        ba + (chk.key + chk.query)),
                       chks[(current_depth - 1) * CHK_PER_INODE:][:pt_elements],
                       pt_block)
            pt_size = pt_elements * (CHK_HASH_SIZE + CHK_QUERY_SIZE)
        assert (len(pt_block) == pt_size)
        assert (pt_size <= DBLOCK_SIZE)
        off = compute_chk_offset_(current_depth, read_offset)
        logging.debug("Encoding data at offset " + repr(read_offset) + \
                      " and depth " + repr(current_depth) + " with block " \
                      "size " + repr(pt_size) + " and target CHK offset " + \
                      repr(current_depth * CHK_PER_INODE))
        pt_hash = sha512_hash(pt_block)
        pt_aes_key = AESKey(pt_hash)
        pt_enc = aes_encrypt(pt_aes_key, pt_block)
        pt_enc_hash = sha512_hash(pt_enc)
        chk = Chk(pt_hash, pt_enc_hash)
        chks[(current_depth * CHK_PER_INODE) + off] = chk
        if (0 == current_depth):
            read_offset += pt_size
            if (read_offset == size) or \
                    (0 == (read_offset % (CHK_PER_INODE * DBLOCK_SIZE))):
                current_depth += 1
        else:
            if (CHK_PER_INODE == off) or (read_offset == size):
                current_depth += 1
            else:
                current_depth = 0
 
 

References aes_encrypt(), compute_chk_offset_(), compute_depth_(), compute_iblock_size_(), min, and sha512_hash().

Referenced by chkuri_from_path().

Here is the call graph for this function:

Here is the caller graph for this function:

chkuri_from_path()

def gnunet-chk.chkuri_from_path

(

path

)

Returns the CHK URI of the file at the given path.

path: the path of the file whose CHK has to be calculated

Definition at line 343 of file gnunet-chk.py.

def chkuri_from_path(path):
    """Returns the CHK URI of the file at the given path.
 
    path: the path of the file whose CHK has to be calculated
    """
    size = os.path.getsize(path)
    readin = open(path, "rb")
    chk = compute_rootchk(readin, size)
    readin.close()
    return chk.uri()
 
 

References compute_rootchk().

Here is the call graph for this function:

usage()

def gnunet-chk.usage

(

)

Prints help about using this script.

Definition at line 355 of file gnunet-chk.py.

def usage():
    """Prints help about using this script."""
    print(
        """
Usage: gnunet-chk.py [options] file
Prints the Content Hash Key of given file in GNUNET-style URI.
 
Options:
    -h, --help                : prints this message
"""
    )
 
 

Referenced by dns_string_to_value().

Here is the caller graph for this function:

Variable Documentation

DBLOCK_SIZE

tuple gnunet-chk.DBLOCK_SIZE = (32 * 1024)

Definition at line 33 of file gnunet-chk.py.

CHK_PER_INODE

int gnunet-chk.CHK_PER_INODE = 256

Definition at line 39 of file gnunet-chk.py.

CHK_HASH_SIZE

int gnunet-chk.CHK_HASH_SIZE = 64

Definition at line 41 of file gnunet-chk.py.

CHK_QUERY_SIZE

int gnunet-chk.CHK_QUERY_SIZE = CHK_HASH_SIZE

Definition at line 43 of file gnunet-chk.py.

GNUNET_FS_URI_PREFIX

string gnunet-chk.GNUNET_FS_URI_PREFIX = "gnunet://fs/"

Definition at line 45 of file gnunet-chk.py.

GNUNET_FS_URI_CHK_INFIX

string gnunet-chk.GNUNET_FS_URI_CHK_INFIX = "chk/"

Definition at line 47 of file gnunet-chk.py.

opts

gnunet-chk.opts

Definition at line 370 of file gnunet-chk.py.

Referenced by OIDC_parse_authz_code(), set_listen_cb(), and task_start_reconcile().

args

gnunet-chk.args

Definition at line 370 of file gnunet-chk.py.