The prime concert of robust a peer-to-peer mesh software is security. On this page, we present a quick overview of the security mechanisms used by this software. The basic goal is to guard against rogue nodes or outside network attackers from doing bad things to the rest of the mesh network, the security goal is not however to protect against a local attacker that has physical access to the computer and/or obtains access by other means. However, an attacker with local access and control of one node must not be able to gain inappropriate access to other nodes.

Information distribution system

To start with, each computer starts by creating itself a self-signed certificate, this certificate will be used by a node to authenticate to other nodes. Each node has a node identifier which is the hash of the certificate public key of the node. So given only a node identifier, we can verify that data signed by the node is authentic. Each node generates information about itself, signs it and sends it out to other nodes. This information block contains the current IP address of each network interface, computer name and other peer-to-peer book keeping information. This signed information block is send to nearby nodes which in-turn may send it out to other nodes in the network, each time a node receives information about other nodes, it checks that the information block is correctly signed and was not modified. Each information block contains a monotonic counter or serial number that only increments each time a updated block is signed and send out. All nodes in the mesh network attempt to only keep the signed information block with the highest serial number for each node they encounter.

This PKI system along with the monotonic counters are the basis for scalable and secure information distribution throughout the mesh network. Nodes can quickly gets updated information about other nodes without having to contact each and every node in the mesh.

Discovery system

When a node first connects to a network, it has various ways to discover other nodes that are accessible within a particular network. This first technique used is a simple, low TTL multicast on the network to find nodes that may be quickly discoverable. Second, the new nodes may have an old list of node information block it can attempt to use to search for other nodes. Lastly, if the mesh network is configured to use a central web service, the mesh agent will contact the web service to obtain information about possible accessible nodes around it. Once one or a few other nodes are found, synchronization of node information blocks will quickly allow the new node to get accurate information about all of the know nodes on a given network. These techniques are the basic node boot strapping system that is used to build and grow the mesh network. Whatever discovery method is used, discovery is not secure in itself, a node can't assume anything about another node unless that node is first authenticated.

Node authentication

Once IP addresses of other possible nodes are found, we need to perform node authentication. Each node uses it's own certificate to sign a TLS server certificate that is used for running a local HTTPS (TLS) server. Basically, each node runs a HTTPS server with a self-signed certificate as the root. When a node connects to another, it will simply hash the public key of the root certificate of the server to obtain the node identifier it connected too. Since each TLS connection is also mutually-authenticated, both client and server know the node identifier of the other. After some bookkeeping, both nodes trade private keys that will be used for authenticating and encrypting UDP traffic between the nodes. The HTTPS session is dropped and most future conversation between nodes will occur using UDP.

Node synchronization

The mesh software spends most of the time idling in the background of a computer and occasionally checking up & synchronizing with other peer nodes using encrypted and authenticated UDP. If after a few UDP attempts a peer does not respond, the mesh software will attempt to perform TCP node authentication or optionally contact Intel AMT if that target peers supports it. The UDP protocol is build in such a way that only one of the peers needs to keep state about the other. It's possible for a node monitor a peer without the peer doing the same back. The UDP protocol and the relation between peers is not always symmetric. At a given interval of time, a node (A) may send a synchronization start packet to another node (B), this packet is authenticated and encrypted using a key that node A previously obtained from node B. In order to keep no state, node B generates the pre-shared key by hashing a session secret with the node A identifier. This key value can always be re-generated by node B when needed, so it does not need to be stored in a database. It also changes each time node B is restarted.

Node A sends a packets to node B using the key that was obtained from B and within the encrypted UDP packet is a response privacy and authentication key. Node B decrypts and authenticates the packet from node A, and responds to it using the included return key without the need to keep any state about node A. This makes the UDP protocol security yet asymmetric and very flexible. The UDP protocol is used to exchange node information blocks and confirm that other peers are still present on the network.

Certificate store

A mesh node can be compromised if an outside attacker can obtain the node's private certificate key. Usually, local storage of the private key on the disk is perfectly acceptable since it is not within the scope of this project to stop attackers that gain access to a computer using other means or have local disk or memory access. Yet, it is possible in future implementations of the mesh software to protect the certificate private key using operating system services or hardware mechanisms such Trusted Platform Modules (TPM).

Self update

The mesh software agent running on each computer will self update when it find a peer that has a better version of the same software that is also built for the same system architecture and operating system. Much like any other self-updating system, the mesh agent will only replace itself with another agent that is signed by a trusted administrator. A node may download the agent executable of another node by performing an HTTPS request, it must then run a hash of the executable and verify it's integrity. In future versions, it's possible that the new executable would be called with a "-t" parameter to perform a self-test before it's used to replace the existing agent.


This project attempts to provide a relatively high level of security, of course, we can't make any claims and this all depends on the actual implementation of the solution. From an architectural point of view, this design is based on existing cryptographic techniques that are well known and well understood.

Other topics: Concept
Updated: 9/11/2009