Edit made on May 05, 2018 by Lawrie at 08:40:04
Deleted text in red /
Inserted text in green
WM
HEADERS_END
Discussion about the current Luddites ~Prototype being used to explore the design concepts & implications.
At present have:
* core code in python
* using single host KVS rather than DHT
* shell overlay scripts for user interface
In near term need to:
* review modularization and API's between code so can more easily make changes underneath
* switch core to using a true DHT KVS - likely standalone
** proposal is to use the bmuller kademlia python package, see https://github.com/bmuller/kademlia
** this in turn uses his RPCUDP package https://github.com/bmuller/rpcudp
** which provides a python RPC over UDP - using Twisted & MsgPack
** see his doc at http://kademlia.readthedocs.io/en/latest/
* switch core to using standard python crypto library
** most likely NaCl, see: http://nacl.cr.yp.to/index.html
* move user interface code from shell into python, likely distinct module using some extension of existing core
* decide if will have structure for data stored in DHT - largely will just whatever structure we decide on in implementation
** perhaps just encrypted blob or clear certificate
* and if will have transfer syntax / structure for message & other data as well (that are then encrypted)
** will likely be determined by DHT package used, ie if bmuller one above, then get MsgPack serialization of python data
!! Lawrie's Python Code Rewrite (v0.3 Apr 2018)
Lawrie is currently rewriting Colin's original prototype, and his script front-ends, into a single integrated python package and client scripts. All code is being developed in python3, though is also being tested with python2 so works with both versions - which does need some contortions in places. Specifically, the biggest differences is with handling of str, just byte arrays in python2, but are Unicode strings in python3 with bytes as a distinct data type, and en/decoding("utf-8') needed between them (main impact in kvs.py & en/decrypt code). Some std module imports have changed - currently urllib used in kvs.py the most obvious. And import handling has changed, necessitating some sys.path hacking to handle running code both within an uninstalled package, and directly within the dir (see config.py for this).
The key externally accessed modules I've created in the new luddies package are:
* ddm_config - global ddm_config dict
* contacts - Contacts class to manage the user's DDM contacts
* identities - Identities class to manage the user's DDM identities
* messages - Messages class to manage the user's saved messages list
Internally, other key modules are:
* kvs - provides the interface to the KVS (Key-Value-Store)
* pkcrypto - provides the interface to the public-key crypto used to en/decrypt messages (currently just modified EDHMW in pkc_edhmw)
This rewrite is being done under git control with a remote repository at gitolite3@gitolite.ozlabs.org:lpb/luddites
Access to this is via manually configured SSH keys. Lawrie & Colin (soon) have R/W access, and the @ludditestester key (in admin folder in repository) has R only access (so we can give it out to people who want to play with the prototype).
!!! Key data structures
!!!! Contacts (in contacts.py)
Each contact is a dict of the form:
{{{ {handle, alias, description, password, published, certificate, my_handle} }}} where:
* handle - Luddites handle for this contact (@tag)
* alias - local alias for this contact (used by client to show known contacts)
* description - (optional) local description of who this contact is
* password - to access identity details in KVM if protected, or None if public
* published - True/False flag whether identity details published in KVM
* certificate - certificate with alg & key details for this contact, retrieved from KVM if published, entered manually otherwise. Format below.
* my_handle - which of my identities is used to correspond with this contact
The new contacts.py module implements a Contacts class to manage this data, loading/storing it to the user's contacts file (default ~/ddm/contacts.dat).
!!!! Identities (in identities.py)
Identity is a dict with the following details on each of the user's identities:
{{{ {handle, alias, description, password, published, certificate, privkey, lastpoll, nextpollint} }}} where:
* handle - Luddites handle for this identity (@tag)
* alias - local alias for this identity (used by client to show known identities)
* description - (optional) local description of who this identity is
* password - to access identity certificate in KVM if protected, None if public
* published - True/False flag whether identity certificate published in KVM
* certificate - certificate with alg & public key details for this identity. Saved in KVM if published.
* privkey - private key details (alg dependent)
* lastpoll - timestamp in secs when last polled for messages
* nextpollint - num secs until next poll (minpollint - maxpollint)
Certificate is a dict with the following alg & key details for a contact:
{{{ {handle, dropseed, timeslice, alg, pubkey, valid_from, valid_to} }}} where:
* handle - of certificate owner
* dropseed - random dropseed used to send messages
* timeslice - mask used on current time to create discrete timeslot for messages
* alg - string identifying which crypto alg is used for messages
* pubkey - dict specifying public key (alg dependent - {base, modulus, pubkey})
* valid_from - time cert is valid from (None if indefinite)
* valid_to - time cert is valid to (None if indefinite)
The new identities.py module implements a Identities class to manage this data, loading/storing it to the user's identities file (default ~/ddm/identities.dat).
!!!! Messages (in messages.py)
Messages are a list of items with the following details for each message saved:
{{{ [my_handle, remote_handle, polled, time, msg_text] }}} where:
* my_handle - my identity message was sent from/to
* remote_handle - remote identity message was sent to/from
* polled - true if message was polled (ie sent from remote_handle to my_handle)
* time - the time interval the message was sent in
* msg_text - text of message
The new messages.py module implements a Messages class to manage the users saved messages list, loading/storing it to the user's messages file (default ~/ddm/messages.dat).
!!! KVS Data format
Currently on the solipsys web KVS currently in use, data is sent as b64encoded strings/bytes, cleartext except for encrypted messages.
Lawrie has changed the format to JSON (JavaScript Object Notation), which is a language-independent data interchange format. Parsing libraries exist for many languages, including python, so I believe this makes it a pretty portable, text based format. Using this encoding could avoid the need to b64encode the data, since it is already in text form, save that the current script requires it. For more info on JSON see:
* https://www.json.org/
* https://en.wikipedia.org/wiki/JSON
The two main structures we need to support are:
* clear certificate for some handle - should use the certificate structure shown above
* encrypted messages - would encode the tuple (msg_pubkey, encrypted_msg) where both msg_pubkey & encrypted_msg are byte strings (nb. python3 distinguishes bytes from strs, care needed here)
Discussion??