Luddites Prototype

AllPages RecentChanges Links to this page Edit this page Search Entry portal Advice For New Users

At present have:

• core code in python
• using single host KVS rather than DHT
• shell overlay scripts for user interface

• review modularization and API's between code so can more easily make changes underneath
• switch core to using a true DHT KVS - likely standalone
• proposal is to use the bmuller kademlia python package, see https://github.com/bmuller/kademlia
• this in turn uses his RPCUDP package https://github.com/bmuller/rpcudp
• which provides a python RPC over UDP - using Twisted & MsgPack
• see his doc at http://kademlia.readthedocs.io/en/latest/
• switch core to using standard python crypto library
• most likely NaCl, see: http://nacl.cr.yp.to/index.html
• move user interface code from shell into python, likely distinct module using some extension of existing core
• decide if will have structure for data stored in DHT - largely will just whatever structure we decide on in implementation
• perhaps just encrypted blob or clear certificate
• and if will have transfer syntax / structure for message & other data as well (that are then encrypted)
• will likely be determined by DHT package used, ie if bmuller one above, then get MsgPack serialization of python data

Lawrie's Python Code Rewrite (v0.3 Apr 2018)

Lawrie is currently rewriting Colin's original prototype, and his script front-ends, into a single integrated python package and client scripts. All code is being developed in python3, though is also being tested with python2 so works with both versions - which does need some contortions in places. Specifically, the biggest differences is with handling of str, just byte arrays in python2, but are Unicode strings in python3 with bytes as a distinct data type, and en/decoding("utf-8') needed between them (main impact in kvs.py & en/decrypt code). Some std module imports have changed - currently urllib used in kvs.py the most obvious. And import handling has changed, necessitating some sys.path hacking to handle running code both within an uninstalled package, and directly within the dir (see config.py for this).

The key externally accessed modules I've created in the new luddies package are:

• ddm_config - global ddm_config dict
• contacts - Contacts class to manage the user's DDM contacts
• identities - Identities class to manage the user's DDM identities
• messages - Messages class to manage the user's saved messages list

• kvs - provides the interface to the KVS (Key-Value-Store)
• pkcrypto - provides the interface to the public-key crypto used to en/decrypt messages (currently just modified EDHMW in pkc_edhmw)

Access to this is via manually configured SSH keys. Lawrie & Colin (soon) have R/W access, and the @ludditestester key (in admin folder in repository) has R only access (so we can give it out to people who want to play with the prototype).

Key data structures

Contacts (in contacts.py)

Each contact is a dict of the form:

 {handle, alias, description, password, published, certificate, my_handle} 

• handle - Luddites handle for this contact (@tag)
• alias - local alias for this contact (used by client to show known contacts)
• description - (optional) local description of who this contact is
• password - to access identity details in KVM if protected, or None if public
• published - True/False flag whether identity details published in KVM
• certificate - certificate with alg & key details for this contact, retrieved from KVM if published, entered manually otherwise. Format below.
• my_handle - which of my identities is used to correspond with this contact

Identities (in identities.py)

Identity is a dict with the following details on each of the user's identities:

 {handle, alias, description, password, published, certificate, privkey, lastpoll, nextpollint} 

• handle - Luddites handle for this identity (@tag)
• alias - local alias for this identity (used by client to show known identities)
• description - (optional) local description of who this identity is
• password - to access identity certificate in KVM if protected, None if public
• published - True/False flag whether identity certificate published in KVM
• certificate - certificate with alg & public key details for this identity. Saved in KVM if published.
• privkey - private key details (alg dependent)
• lastpoll - timestamp in secs when last polled for messages
• nextpollint - num secs until next poll (minpollint - maxpollint)

 {handle, dropseed, timeslice, alg, pubkey, valid_from, valid_to} 

• handle - of certificate owner
• dropseed - random dropseed used to send messages
• timeslice - mask used on current time to create discrete timeslot for messages
• alg - string identifying which crypto alg is used for messages
• pubkey - dict specifying public key (alg dependent - {base, modulus, pubkey})
• valid_from - time cert is valid from (None if indefinite)
• valid_to - time cert is valid to (None if indefinite)

Messages (in messages.py)

Messages are a list of items with the following details for each message saved:

 [my_handle, remote_handle, polled, time, msg_text] 

• my_handle - my identity message was sent from/to
• remote_handle - remote identity message was sent to/from
• polled - true if message was polled (ie sent from remote_handle to my_handle)
• time - the time interval the message was sent in
• msg_text - text of message

KVS Data format

Currently on the solipsys web KVS currently in use, data is sent as b64encoded strings/bytes, cleartext except for encrypted messages.

Lawrie has changed the format to JSON (JavaScript Object Notation), which is a language-independent data interchange format. Parsing libraries exist for many languages, including python, so I believe this makes it a pretty portable, text based format. Using this encoding could avoid the need to b64encode the data, since it is already in text form, save that the current script requires it. For more info on JSON see:

• https://www.json.org/
• https://en.wikipedia.org/wiki/JSON

• clear certificate for some handle - should use the certificate structure shown above
• encrypted messages - would encode the tuple (msg_pubkey, encrypted_msg) where both msg_pubkey & encrypted_msg are byte strings (nb. python3 distinguishes bytes from strs, care needed here)