Designing Nodes and Arrows in Knowledge Graphs with Semantic Spacetime

The surprisingly hard part of knowledge representation (for advanced readers)

There are only two hard things in Computer Science: cache invalidation and naming things.

— Phil Karlton

This is an article about a rather difficult topic: modelling knowledge, i.e. how we conceptualize knowledge to make it both accessible and searchable, so that it tells us the stories we want to hear. After half a century of information technology, you’d think we’d have cracked this by now, but instead we’ve become stuck in a few patterns (typically constrained by habit and commercial influences). That’s normal, but we can still do better. For the Semantic Spacetime project, the challenge is this: how do we join together individually meaningful components into a knowledge map that actually helps us–not only to remember details, but also–to think. Knowledge graphs are one method, but conventional models are very poor. How do we do better?

One of the hardest things about designing a knowledge graph for its users lies in turning our own intuitive and free-flowing thoughts and ideas into the kind of formal relationships that can be searched by a computer–whether for ourselves or for others later. I covered this point in Why Are We So Bad At Knowledge Graphs? A key design issue there lies in choosing appropriate text, or naming things, so that what we are going to read in the future accurately reflects what we intended in the past. When we rewrite ad hoc language into unfamiliar arrow-relationships there is an additional burden involved.

This transformation from ad hoc thoughts to a formal representation is desirable mainly for the purpose of automating the look up. It’s probably not something many of us can easily write down in one take. Predicting a future state of mind is hard. Predicting others’ states of mind is harder still. In general, tha art of pedagogy is hard. Not everyone makes a good writer or a good teacher. It usually takes several attempts, with review, revision, reworking, and trial-and-error patience to get the message right. Even then, it’s still possible to get it wrong.

The devious siren of human language

British ship officer: Mayday! Mayday! This is the battleship Valiant! We are sinking! I repeat, we are sinking!

German coms officer: Hello, Valiant, nice to meet you. What are you sinking about?

(Recalled vaguely from a Norwegian television advertisement for a language school)

With the rise of Large Language Models, and their ability to generate convincing language that feels trustworthy, many researchers and users have abandoned the logical approach and have embraced “free speech”, i.e. so-called natural language query response. But, this is always going to be a difficult problem. We want all the convenience and familiarity of natural language, but with all the reliability of precise logic. But this is naive–those two goals tend to pull in different directions. One can’t rely on “natural” human language to express logical distinctions, because it is (by its nature) built on metaphor and borrowed ideas rather than on precise one-to-one mapping of concepts.

A possible solution is the approach employed by the Semantic Spacetime model. The idea is to fit actual meaning first to a simple skeleton of concepts about how processes unfold, and then to add flesh to those bones by decorating the skeleton with descriptive natural language phrases for familiarity. This is the approach developed over a series of papers and applied in a knowledge graph project SSTorytime.

Gradually build trust or cut straight to the logic

We have to make a choice when capturing ideas: do we want the final form of our searchable notes to be easy-going and natural (but possibly unclear or less precise) or do we want them to be ultra-precise and error free. The latter might not be even achievable, but it feels like a desirable goal. It’s something we might strive for.

Logic and mathematics are specially invented special languages that offer precision. They are deliberately concise — perhaps writing just a single symbol X along with special symbols for operators etc, instead of using a long sentence or several words as a philosopher might. We use special symbols and special jargon in order to be both specific and concise, stripping away feelings and vagaries to leave just this essence. The unfamiliarity is both a benefit and a curse. It’s a benefit because it avoids muddling ideas with words we think we know the meaning of (but might misconstrue), but it’s a curse because you have to learn a new lexicon of concepts and words/symbols for each case.

Less is sometimes more. But like distilling a fine whiskey, making a perfume, or reducing the perfect sauce, it takes time and effort to boil something down to its perfect essence. It’s not an easy path to take. If you take the time, it’s hard work. If you don’t bother, the result may or may not serve its intended purpose. This translation to an unfamiliar language is why people find mathematics difficult, and similarly a reason why IT projects, and in particular knowledge projects fail.

• People find the abstracting of an essence difficult. Handwaving is easier.
• We fail to simplify language (because that requires a discipline of additional work)
• We may lose our audience or fail to engage others because of this formality.

The problem with our natural handwaving skills is that handwaving only works in conversation, when there is a receiver who shares our train of thought and can make inferences in real time, by mirroring our own intent. We can handwave and they will fill in the gaps for us. This has the added function of being trust-building, so common language works on a number of levels. It’s not only about information. However, computers are dumb and they can’t do this. Even Large Language Models (enormous in size and unsustainably expensive to run) don’t do this very well. That means we have to put in the work in advance to reduce our ideas to a kind of “universally understandable” form, but by doing so we risk derailing the effort to capture knowledge. Mathematics and logic do this by inventing a very simple and reliable language–but only for those who dedicate their lives to them. They feel unnatural to many.

If knowing something well means “knowing it like a friend”, then clearly the barrier to passing on knowledge is a lack of familiarity. We need to be able to think about ideas in our own way, while still adding some discipline to them as an investment in future searchability.

Naming is hard, because of the baggage

We would like the relations in a graph to read like natural language as possible. How far this is possible depends on the language. When all we have in a graph is nodes (or vertices) and arrows (links or edges, or whatever we want to call them) then we need to understand carefully how we name these components. Choosing the text for nodes and naming arrows might be the biggest challenge in formalizing thoughts and ideas. When we use natural language, words carry all kinds of connotations, or possibly unwanted baggage with them. We need, effectively, to design an algebra. It’s worth spending a few lines to understand why language is unreliable.

Let’s say we want to turn some thoughts into a map of things with arrows between them–i.e. a knowledge graph. It’s schematic and relatable, but not necessarily precise. When we use arrows intended for ourselves, we want to quickly be reminded of the intended meaning. When we use arrows for others, we want arrows to have a universal appeal to everyone and even to encourage people to think like us.

In modern Western thinking, we are used to short names. Before we had a simple standard for names, people used to be called after their family histories, and entire addresses. Thor, Son of Odin, wielder of magical hammer, smithy of 17 Yggdrasil Street, Hall of the Mountain King. Since that’s a bit long, it gets abbreviated to Thor Smith. However, in abbreviating the name, we lose a lot of information. When writing knowledge for a knowledge base, there is no need to abbreviate everything, because it costs us basically nothing for the computer to show the full story. Similarly, we try to use short identifiers when storing information. This is not the right approach for capturing knowledge.

A goal of the SST is to reduce this sensitivity to naming, so that it matters less whether we choose the ideal names to tell the best possible story, because the underlying SST relationship types will still find the paths. This is the benefit of relying on generic process structure (spacetime concepts) and not on logical ontology. Process is forgiving, but logic isn’t.

Free language is unreliable

So let’s consider how to adopt good practices in writing notes. Nobody starts out thinking about how to express their experiences or intentions in terms of abstract logical relationships. But one of the goals of formalization is to condense, distill, and simplify ideas using language into a kind of standard lingua franca that potentially anyone could understand.

Consider a few examples first of how language serves us poorly when we try to abbreviate it. Suppose we want to say that “X expresses property Y”. This has two meanings in English :

• Before our very eyes, X displays or announces some skill or attribute Y for others to see (Mark is doing a good job!).
• The nature of X is such that its objective property Y is manifestly associated with X at all times (Mark has blue eyes).

The first of these is about a particular event. The second is an unchanging (invariant) property of X. These are very different things, but we can use the same word “express” in different ways. In common knowledge graph models, these two meanings are easily muddled together, because simplified language doesn’t capture the distinction without a lot of additional words. Modern information technology, from encyclopaedias to databases, has shaped us to think about knowledge as a kind of snapshot of unchanging information, rather than as a collection of stories. Yet, we need both points of view, because much of our knowledge comes from episodic encounters that we (only later) dissect to extract general and lasting characteristics. Another example is found in these two clipped sentences:

• Person studies history.
• Person studied history.

The first case is either a single episode, event, or collection of ongoing events. The latter is something that happened in the past and became a property forever associated with the person (we are each our own histories). We begin to see a problem in taking words at face value.

The distinction of events versus generalities is obvious when pointed out, but language does not immediately make the distinctions obvious. Here’s another case:

• Server X authenticates client Y.
• Servers authenticate clients.

Which of these are we intending to represent by a graph? The first of these is a single event. The second is a generic statement about attributes (server “promises” if you will). In the first example, Server X/Client Y are specific computer processes, in the second Servers/Clients represent the concept of a server and a client, not an actual thing. The way computer science approaches this is to employ type systems. We might write

X (authenticates) Y

Where X is of type server and Y is of type client. Now we can make a rule about servers and clients, that we call an ontology. But–this leads to an exponential explosion of concepts and rules that can never be completed.

This distinction between the concept of something and a specific realization of the idea leads to a lot of confusion, because the two have different semantics–and this is reflected in the different usage of “authenticates”. In the first case, it is an action. In the second case it is a generic behaviour, which is a property attribute.

Language is wonderful and amazing, but it is not a simple machine. It’s a tool in a remarkably involved process that involves inferences all the way down. The key problem is that we muddle concepts together all the time on purpose. For all the focus on grammar and rules of language, I suspect that grammar plays only a minor role in elucidating meaning.

Semantic Spacetime’s (3,4) approximation

The SST approach tries to base different meanings on a model that uses 3 kinds of “graph node” and 4 kinds of link. This is called the gamma(3,4) representation, with 3 kinds of agent and four kinds of relation. Gamma(3,4) provides a skeleton for process representations in space and time that can be renamed any number of times for specific scenarios, but which captures the universal meaning. If that seems like an odd idea, it turns out that most of our meanings are just metaphors for spacetime ideas. After all, deep down there is nothing else to know than what is up down left right forward behind along with colours and other attributes.

The three node types are: events, things, and concepts (etc). The four link types are

1. Change events or transitions, about ordered successions of states (LEADSTO)
2. Hierarchies of containment, composition, or configuration (CONTAINS) things inside other things.
3. Steady state properties or invariant behaviours (EXPRESSES PROPERTY) descriptive concepts.
4. A judgement of similarity or equivalence, either physical or virtual (NEARTO).

The claim is that these are sufficient to represent the basic meaning of any scenario. Even with this approach to simplifying, it may not always be easy to find the right (3,4) model. The language muddle will always be there, but we can try to resolve it by careful selection from this set. In the examples above, the first of the pairs (publishes/expresses etc) are of type 1 between events: a causal action that happens in an isolated event. Then the second items of each pair are of type 3 between concepts. Because this task is not trivial, the SSTorytime project comes with a library of suggestions as a guide.

A simple way to think about it is to imagine everything we want to say as a scene to be described, so that an effective SST knowledge map acts like a Scene Description Map or Language.

Mapping out universal characteristics of a scene

Let’s analyse an example. In an earlier post Why are we so bad at knowledge graphs? I used an example of a passage from Alice in Wonderland to illustrate how our thinking goes wrong. Let’s study that example in detail. We start out with the original passage:

Alice did not feel encouraged to ask any more questions about it, so she turned to the Mock Turtle, and said, “What else had you to learn?”

“Well, there was Mystery,” the Mock Turtle replied, counting off the subjects on his flappers, — “Mystery, ancient and modern, with Seaography: then Drawling — the Drawling-master was an old conger-eel, that used to come once a week: he taught us Drawling, Stretching, and Fainting in Coils.”

Suppose we find this interesting for some reason and we want to remember it, perhaps even write notes about what it means, by turning it into a graph. But what does that mean? We might try to capture two things about it:

• The quote itself.
• The meaning of the quote to us and to others.

But we are faced with a barrier: representing this as a graph. Why would we want to do that? A graph is a map of some process, not a way of naturally expressing language.

Many folks will read a tutorial about knowledge graphs and start out by doing something unnatural, like this:

Or something even less helpful trying to use the infamous “is a” relation to classify everything in species:

This is all very clumsy and unhelpful.

Knowledge doesn’t only come to us as facts. It arrives in events embedded within scenes–that’s because we experience knowledge, whether directly through our senses or indirectly when reading a book. If we don’t succeed in capturing the experience of learning, then the facts have no support or context by which to recall them later. The key to recall is lost. This should inform the way we try to express knowledge–and it’s particularly important for knowledge graphs. No matter how we choose to express our intended meanings through words in our notes, knowledge has a certain structure (a certain shape) that guides us in understanding the meaning. That shape is based on what a text mean in terms of space and time. I’ve written about this before, in connection with the SSTorytime knowledge graph.

Is it about nouns and verbs?

When we read about knowledge graphs, authors often offer the simple rule that nodes are nouns and that arrows are verbs (this is not correct). It’s a lovely idea that it could be so simple, but it’s not. Another mistake lies in the desire to turn knowledge into mere data for a database, by stripping it down. We want to think about this as if it were a kind of quasi Normal Form, with no repeated text elements. All this leads to unhelpful advice. We are often trying too hard to isolate ideas from one another and create a kind of “normalized database” (3NF) representation of scenes. This is like trying to reconstruct a person by taking one gene for each desired characteristic. Alas, that’s not how it works.

The first reason is that repetition is an important part of reinforcement and explanation in intent, so we shouldn’t be afraid of repeating words. The second, and more important problem, is that it is ambiguous and somewhat incomprehensible to strip away the parts of language that generate appropriate inferences. How do we proceed from here? It sort of follows the forms of algebra or logic more than the original text, but what value does it actually add?

Perhaps the most important fault of this approach is that it is trying to take a sequence of events and flatten it into a set of invariant and timeless relationships, as in the cases above. That’s an approach that might work for invariant data like telephone numbers in a directory or book titles in a library, but this is not how we think about episodes in our lives, or even episodes described in books. Language works by reusing atoms in combination. If we tried to have a unique word for every different concept it would be impossible to remember.

Timeline first

Spacetime comes to the rescue and helps to sort out the mess. The first step in decomposing something is to establish the order of events in a timeline. Forget about nouns and verbs and think in terms of pictures. Sometimes timelines branch or converge, but this example is quite simple:

Each of these nodes is like a frame in a movie. A sequence of scenes in a script. We understand this easily, but what are the arrows? In this case, we could simply call them “then” to emphasize that they form a sequence. We could get more fancy and call them “so therefore” and “then out of the blue” etc, to add more storytelling flair, but basically they are causal order arrows “then/next”.

If we were thinking about some kind of “third normal form” approach to data representation, this would seem wrong. “Why are we repeating Alice three times. Alice should only appear once. She is just a noun.” But this is wrong thinking. Alice is not a statue, frozen in time. The Alice in the first event is not the same Alice as in the second event. She (the state of Alice) has moved on. The concept of Alice is the same in each frame, but it’s not the concept of Alice that feels or speaks.

When we try to oversimplify knowledge by reducing it to proper names, we misunderstand the essence of what communication is about. Yes, there is abstraction of concepts, but there are many versions of Alice to consider: the physical person at one moment, the physical person later, a photograph of Alice, the invariant concept of Alice (which lives on even after she is dead), and so on. If we want to make sense of the world, we cannot muddle up these things we know to be true by botching the data model.

Scene description representation

What this decomposition does is reduce the text to a number of scenes. We can now use other kinds of links to describe the scene. We can try to reduce each scene using more formal expressions, e.g.

Reading around the text from outside this passage, we learn more about the scenes, and we can add more notes.

Finally we can think about where this all takes place. We could draw a ring around all of these events and say that they all occurred at a particular location or as part of a chapter in the book (or indeed both). The point of annotating is, after all, to add all useful information about each scene.

Another container might be “Wonderland”. We can go on adding to this graph. The essential point is that this is all quite easy to understand, without any major linguistic contortions or unnatural arrows. That’s because we’ve identified arrows with processes that we know well from what happens around us–in space and in time.

The contains category is perhaps the least flexible, perhaps because it’s a trivial trick, like the infamous “is a” relation. Saying that a thing “is a” something else is particularly trivial–it’s just giving it an alias. A rhino is a rhinoceros shaped animal. Like saying “Would you like a bag for your shopping”. It doesn’t change anything or generate any new insight.

This is perhaps why containment is a relatively underrepresented class of relations. It expresses only names for ad hoc collections or focus groups. An exception is when membership in a group or restriction to a region asks how the things inside are equivalent in some sense — and thus it has a relationship to near/similar for virtual distance or physical distances enclosed by regions. Patchworks can always be extended or reduced to fit demand. The result explains why taxonomies are generally useless constructions…while common knowledge by social norms is a better model.

Why events are a blessing not a curse

When we take someone’s photo or paint their portrait, it’s no longer them, but just a reminder about them. A representation of the concept of them. Nothing about the physical picture is them, only their idea remains. In everyday usage this is all just fussy philosophical nonsense, but it matters to machinery because our brains know the differences in spite of them all having the same name.

A difficult idea for many is the idea that a character in a play is not an invariant data record, but a collection of events: a timeline or a personal history. The person might express an invariant property: their name, for instance. But they are not their name.

We can learn a bit from Einstein in this regard: different observers see things differently. Locations and times are all part of the journeys we undertake in our paths through life. A thing or person is really a world line–a thread or process through space and time. Some of the attributes change, and some remain the same.

Things get more interesting though, when agents interact intentionally. This is the domain of Promise Theory. Consider the following: we might want to express a concept like “X buys Y”. This has the feeling of an arrow relation, particularly if we fall for the simplistic idea that arrows are verbs and nodes are nouns (please don’t). Going back to the distinction between concepts and events, this sentence actually describes an event–a single happening, not an eternal fact to be consigned to the history books. In this we can simply write a single node in the timeline as:

X buys Y

As a single node text, which is an event. It can be followed by another.

The shop closes.

This is also an event, and because the verb is transitive it looks like a dangling arrow

The shop (closes) ??

You might feel the compulsion to add something on the right hand side to make it a triplet, but this is trying to shoehorn the model’s foot into a badly fitting shoe. The myth of nouns and verbs prevents us from seeing the actual structure of the event, which is like the Alice example. The actual process graph may be decomposed like this by separating time (left to right) from attributes (up down).

The timeline continues from event to event, as an axis running through the episode, because these are not invariant characteristics, they are merely episodes or happenings.

X wakes up (then) X goes to town (then) X buys chips (then) X goes home.

Not every “verb” makes sense as an arrow in a story, or becomes a universal invariant. Most of us wouldn’t want “he bought chips” on our tombstones. Consider:

X picks his nose

Was this a lifetime obsession or a single event?

X picked his nose

If we want to make a kind of rule?

X shouldn’t pick his nose

Then this is a statement with no obvious arrow. Don’t fight it! The intention is captured by the intent. If we need to annotate with arrows, we could say something that relates it to its meaning:

X shouldn’t pick his nose (intended as advice to) X

By restating a sentence, we can change its typological character:

X (affects) Y // X may be an invariant property of a thing or a single event

X (affects) the state of Y // the state of a thing is a temporary event

X (affects the state of) Y // is this a generic comment?

Now, by changing the arrow to a property arrow, X has the property that it can affect the state of Y generically at some time or other, but not a specific time

We could find a way of writing something like this:

S = "X founded company Y"

S (has founder) X

S (has company) Y

S (event meaning) founding is when someone establishes the legal registration of a company

The discussion of ownership is a similar case. Without getting technical, we simply treat the event as an event.

X owns Y

If this is permanent and tombstoneworthy, then we could say that the property Y becomes an attribute of X or X’s extended being. If this it was a temporary situation, then it would be better to write

S = X owned Y for a time

S (role owner) X

S (role property) Y

S (refers to time interval) August 1 to September 12 2024

These are all perfectly legal node labels in a semantic spacetime. A node label could be half a page of text if that helps. If you are using a database that insists on a type model with simple short identifiers, then finding a pliant type model might be quite hard. We don’t need that. The types, if we need them, are implicit in the arrows as long as we define arrows carefully according to the four types.

Seems too complicated?

We see that an accurate depiction of a scene is much more complicated than the simplistic cartoon sketch of our first version (figure 1). Is it now too complicated? That depends on what we want to do with the knowledge. Knowledge is complicated. Only our aversion to investing effort leads to the desire for simplicity. That’s an indication that we don’t really want to know. If we did, no amount of effort would be too much.

With proper annotation, we can now ask questions like: what was Alice trying to do (what was her intent)? What was said? Tell me what happened with the Mock Turtle, and so on. We need to organize knowledge around what the reader is thinking, not like stamp collecting.

When we lay out the story in this way, starting with the events and describing the events in terms of repeated persons, concepts, ideas etc, by factoring and refactoring the sentences, the goal is not to replace the original text, but to add to it. The thing that binds Alice and the Mock Turtle and the concept of learning together is a series of events. That information is useless trivia unless those events can tell us more. Imagine you are a crime scene investigator. You don’t care that there is a body lying on the grass, you want to know how it got there and what the intention was.

We know intuitively what kind of arrow refers to an event, a property of the event, or an object, a person etc. Nevertheless, it’s sometimes easy to get confused by natural language because of the way we re-use words in different ways. These two expressions are different:

Alice (expresses) discouragement // Alice (leadsto) expression of discouragement

Alice (feels/expresses property) discouragement // Alice (expresses property) look of discouragement

In the first case, Alice frowns and we are emphasizing causal responsibility. A look of discouragement was something Alice did intentionally to cause a signal. It’s an active interpretation that caused a change in the scene. In the second case, this is a passive property of Alice in this scene. Alice does nothing but stand passively in the scene. It’s a background issue, perhaps even unintentional and meant as an observation rather than a new event. In both cases, we can use the word “expresses” but in different ways. This is a trap to watch out for, and it’s why Semantic Spacetime asks us to think carefully about what type of arrow we mean. The interpretation for the scene may be quite different. We won’t always get these details right in the beginning. This is why we need to revise, rework, review and repeat.

Representation without idealization is against the constitution!

(apologies to Tom Baker)

Not everything is as simple as we would like it to be. This is why we settle for idealizations in science and in life. Finding the appropriate level of idealization is an art, not a science. Everyone judges for themselves. What is important is to represent our intent properly, and not be tricked into incorrect practices by rules that don’t really fit. We need to stand up for our own intent. Only we know what that is

There is a tendency to want to make lots of different kinds of arrows to feel that we are expressive. But we don’t need to show off our language skills. We are trying to reduce and condense into a universally acceptable and simple form. This may take several rounds of work because we will naturally use certain different words for certain cases, even though we could just use the same words. Clearly much of the challenge lies in designing arrows well. If we want to compose a connected structure that is what we should be thinking about. There is nothing natural about graphs for representation unless you have a specific process in mind.

There are some hard choices to make, but they have to be made to simplify. For example, we want to think of a person as causing something to happen. But then we have to distinguish between the invariant person and the person in a particular moment, which introduces new problems. It’s more complicated, but more accurate, to say that there was an event in which the person (who is an attribute of the scene) caused that thing to happen (that thing also being an attribute of the scene). The way we talk about active processes and events is different from the way we talk about things or concepts and their properties. The distinction might be subtle, but these details (however small) signal real differences in meaning.

In a recent update to the SSToryline project, I moved to a shared dictionary model for arrows to help standardize SST graphs and to ease the difficulty of translating our thoughts into formal language.