Semantic Spacetime and Data Analytics

Figure 1: A local sample of Internet structure from CAIDA data.

CAIDA in Semantic Spacetime

Figure 2: All socio-technical networks are multi-scale graphs of complex overlapping processes.

Semantic Spacetime Again

Figure 3: Probing the local structure of the Internet using traceroute (see part 7 of this series).
Figure 4: Building up a traceroute as radial slices taken over multi-slit maps forms a superposition of possible boundary conditioned paths.
Watch a video introduction to the method for this article

Isn’t it just a graph database?

Hybrid graphs combine local and global indices

Figure 5: In a traditional data warehouse a tombstone or legend marks each location, but we need an index to find each body. A graph is like a treasure hunt, each node pointing with clues to the next.

Internet tomography with ArangoDB

Figure 6: A semantic spacetime model for relation semantics for the elements of the ITDK model. This is a timeless representation, so it fails to capture to full dynamics — yet this is a step towards integrating different observations from traceroute, BGP, and DNS.
THE BASIC MODEL:* <Devices>: locations known to be single bits of machinery. A device may have multiple IP addresses. these are labelled N1, N2, …    <device> HAS_INTERFACE (+expresses) <ip>    <device> ADJ_NODE (+near) <device>    <device> ADJ_UNKNOWN (+near) <unknown>* <Unknown>: an unknown intermediary involved in tunnelling from one device to another, e.g. a switch, repeater, or MPLS mesh. This has an anonymous IP address, so we have to make up a name for it.    <device> ADJ_UNKNOWN (+near) <unknown>    <IPv4/v6>: addresses expressed by <devices>    <device> DEVICE_IN (-contains) <country)    <region> REGION_IN (-contains) <country>* <AS>: A BGP “Autonomous System” entity consists of many devices, may span multiple regions, but non-overlapping    <device> PART_OF (-contains) <AS>* <DNSdomain>: BIND entity consists of many IP addresses, may overlap    <DNSdomain> HAS_ADDR (+expresses) <ip>

Modelling is always based on the answering of particular questions rather than trying to capture an absolute representation of reality as we believe it to be (our idea is based on a subjective point of view).

Figure 7: The aggregation of nodes into cellular grains offers a view based on scaled spatial regions. these have socio-technical significance with respect to different processes and applications.

Quick wins

Figure 8: A log-log plot of internet node Near adjacency degrees (log k, log N(k)) showing the approximate power law for preferential attachment by counting of inferred adjacencies.
// Show IP addresses for a Device N1FOR ip in Contains FILTER ip.semantics == "HAS_INTERFACE" && ip._from == "Devices/N1"  RETURN ip._to// Show Regions within 40,000 km of x,yFOR reg IN Region FILTER GEO_DISTANCE([4.60971, -74.08175],reg.coordinates) < 400000000 FOR dev IN Contains FILTER dev.semantics == "DEV_IN" FILTER dev._to == reg._id RETURN dev._to// Show Devices within 40,000 km of x,y = 4.60971, -74.08175FOR reg IN Region FILTER GEO_DISTANCE([4.60971, -74.08175],reg.coordinates) < 400000000 FOR dev IN Contains FILTER dev.semantics == "DEV_IN" FILTER dev._to == reg._id RETURN dev._from// Show IP addresses within 40,000 km of x,yFOR reg IN Region FILTER GEO_DISTANCE([4.60971, -74.08175],reg.coordinates) < 400000000 FOR dev IN Contains FILTER dev.semantics == "DEV_IN" && dev._to == reg._id FOR aif IN Contains FILTER aif.semantics=="HAS_INTERFACE" && aif._from == dev._from RETURN aif._to// Show neighbours of N2FOR a in Near FILTER a.semantics == "ADJ_NODE" && a._from == "Devices/N2" RETURN a._to// Shortest path from N2 to N3 FOR v, e IN ANY SHORTEST_PATH 'Devices/N2' TO 'Devices/N3' Near RETURN [v._key, e._key]"// Show geo-locations for N2FOR p1 IN Contains FILTER p1._from == "Devices/N2" FOR p2 IN Contains FILTER p2._from == p1._to RETURN { source: p1._to, region: p2._to, country: p2._from }// Show IP addresses for Device N40957FOR p1 IN Contains FILTER p1._from == "Devices/N40957" && p1.semantics == "HAS_INTERFACE"  RETURN { source: p1._to }// Show DNS domains in N40957FOR p1 IN Contains FILTER p1._from == "Devices/N40957" && p1.semantics == "HAS_INTERFACE" FOR p2 IN Expresses FILTER p2._to == p1._to RETURN { source: p1._to, region: p2._to, country: p2._from }// Show Devices for domain DNS/example_comFOR p1 IN Expresses FILTER p1._from == "DNS/example_com" FOR p2 IN Contains FILTER p1._to == p2._to RETURN p2._from// Show me the hyperlinks/wormholes (unknown routing hubs/fabrics)FOR p1 IN Unknown RETURN p1// Show me the Devices addresses that are linked by Unknown tunnels...FOR unkn IN Contains FILTER unkn.semantics == "ADJ_UNKNOWN" RETURN unkn._from

Next steps, new horizons

@markburgess_osl on Twitter and Instagram. Science, research, technology advisor and author - see Http://markburgess.org and Https://chitek-i.org

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Data Structures

LUCK IS LIKE A BUS ……

The MidTerms on Twitter

Modeling Networks of Networks

You Can Get a Job in Data Science Without Knowing Everything

Simple Yet Effective Data Preprocessing Toolbox

The Simple but Profound Introduction to Expectation

Data Science And Web 3.0: Technological Road To The Future

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Mark Burgess

Mark Burgess

@markburgess_osl on Twitter and Instagram. Science, research, technology advisor and author - see Http://markburgess.org and Https://chitek-i.org

More from Medium

Graphs on the Ground Part II: Knowledge Graphs in the Life Sciences

4. Escape!

Implementing sustainability as business rules through knowledge graphs

A knowledge graph schema of relationships between legislations, clauses indicating obligations or prohibitions for companies, and how this can be transformed into business rules, business processes and implementation arrangements to implement applicable legislations.

Embrace Complexity (Part 1)