What might Promise Theory have to say about Physics?
The Return of PT #20: this time it’s personal!
It’s the 20 year anniversary of Promise Theory, and people remind me about the successes of CFEngine and Kubernetes as examples of how Promise Theory has made an impact in the realm of information technology and even been featured in the Gartner hype cycle. However, from time to time, I’m challenged by someone (quite reasonably) to explain what Promise Theory has to say about physics–my own university phylum, and I like this question–even when it’s posed more as an accusation of heresy than as genuine curiosity.
Promise Theory is not intended as a theory of any phenomenon in physics, but that doesn't mean it has nothing to say about physics. In the beginning, I had no opinion about it: it was more a question of what physics might help us to say about computers and “higher level processes”. However, after 20 years of being pulled back to PT over and over again, I have found that it might actually have quite a lot to offer physics, in principle–although I might not necessarily be the one who is smart enough to figure it all out. After all, the basic principle of Promise Theory is that agents can only promise their own capabilities, and that includes me.
Causal origins
Promise Theory began, for me, as a way to explain processes in Computer Science. But processes are processes, so why just computer science? Quickly, the ideas were seized upon by people in management science, and then extended by its generalities to apply to other areas where behaviours needed to be described in more formal terms. In computer science, it was about understanding the implications of local causality. That’s obviously a topic of interest to physics. In sociological matters (now called sociophysics), it was an attempt to replace the awkward attachment to moral philosophical argument with something more formal and impartial (i.e. mathematical). Sociology and philosophy are fields where authors typically have few (or no) mathematical skills to formalize notions. I wanted to help with that.
As a physicist myself, by training, who only stumbled into Computer Science accidentally, I had to learn a different way of modelling phenomena, and I found the question very interesting. I had just finished writing a book about Covariant Field Theory and another about quantitative methods in Computer Science and it seemed like something I ought to be able to say something about. But it wasn’t as easy as I’d imagined. For a start, the familiar tools of differential equations were basically useless, there was no low level continuity in symbolism, and almost nothing was repeatable, so no statistics. I had to start again from scratch using bits of set, theory, graph theory, and game theory for inspiration. None of them had what I needed, nor did the abstruse and popular Category Theory that theorists fawned over for its long words and infinite concepts.
To begin with, describing a new theory is not easy. I meet many conservative scientists who believe that theories actually belong to physics–as if it has a patent on them. If you have a different kind of theory, and your theory doesn’t either address a problem that mainstream physicists are interested in, or improve on a calculation in QED, you are either a quack or merely a handwavy philosopher. I read a lot of books about the history and origins of scientific ideas and was surprised by how much of what we take for granted was high controversial back then. Now, I’m sort of ambivalent about these attitudes. I have some sympathy for this unsympathetic view about quackery, but in the end I can only say that Promise Theory wasn’t something I tried to invent to solve perpetual motion or faster than light travel–it was an honest attempt to understand a problem in “systems” (in the general sense) where the state of the field was already driven by a belief system that was obviously flawed. The problem was this: what does the world look like if it is built entirely from autonomous agents that have a finite capacity to interact if and when they can?
It took me a year to decide on the term “promise” for the characteristics that agents express. It was a mixed response. People don't think very carefully about ideas, so many assumed that promises are always about the future — they aren't. We make promises about all kinds of things, e.g. I promise I delivered the parcel yesterday. Some used this as an excuse to dismiss it as something that only applies to humans, and thus is pseudoscience (as they assume that all that other sociology is nonsense). I chose it with some defiance because it was general enough to cover all meanings, at least for anyone with an open mind. It could be applied to characteristics that were designed by humans or that were simply ad hoc discoveries, without prejudice. I was aware that certain people (in social sciences and in physics) would object, but basically I didn’t (and still don’t) care about other people’s lack of imagination.
For those good readers who are actually curious, let me try to explain the “bear necessities” of Promise Theory, from the perspective of physics, as I see them. A personal view.
Autonomy = extreme locality
Starting from computer science, I found that (apparently without exception) every journal paper took the view that you could simply tell any system what to do and it would be obliged to do exactly that. There was an appeal to the tradition of Control Theory, from electrical engineering, which is basically about linear deterministic response calculations. In classical physics, we are used to having our way quite accurately. People would attach themselves to this (without ever having done a Laplace transformation) based on the name alone as a license to control anything. Others from a mathematical background would use the notion of obligation as an excuse to use the forms of modal logic to give papers pretensions of theoretical legitimacy.
Around the new millennium, I had a succession of Emperor’s Clothes moments, having designed a very successful system (CFengine) that was based on the diametric opposite idea of every agent for itself, and which had very much proven that the idea of every agent acting independently, worked well. No matter how much you wish to control an “agent” in some process, it might be either unable or even unwilling to comply with external attempts to force it (even merely persuade it). That might not be what you want to hear, but you can’t do much about it except figure out the consequences. Moreover the coherence that one expects from external boundary conditions can also emerge from internal coherence between the agents. Some people started to call that kind of collective behaviour swarm intelligence. The bottom-up "emergent" coherence was both more powerful and adaptive to local constraints than a low information imposition from outside, and every agent could do what it was able to do.
In my view, the natural thing was to appeal to locality as the basic “atomic ground state” of any elementary thing: isolated things had to be causally independent, or autonomous. After all, what an agent can do is limited by the resources and policies ( “rules”, if you must) it has “internally”. No amount of foot stamping or appeals to authority could change that. It seemed just like basic atomism in physics.
But the persuasive belief in obligation is strong within us. To some extent, it also occurs in physics. It goes back to the time of Galileo and Newton, when people believed that there was a god who was omnipotent. The “laws” of physics were very much legal necessities in the style of Moses’ commandments. They were not just statistical patterns, they had to be obeyed on penalty of excommunication. Later, the shocks of quantum theory caused much consternation here, probably because they exposed this idea as a kind of heresy. To divert inquiry even more, Newton and Descartes and others did something else of significance: they separated matter from spacetime and described only the behaviour of matter. Spacetime was to be understood as passive and matter merely moves around within it. It was an obvious choice in an age when physics was about predicting ballistic trajectories, but it has coloured everything that followed. Today, few people question this split, even after the 20th century’s upending of this idea, i.e. talking about quantum fields and gravity that utterly conflate these two ideas. But in Promise Theory there is no “matter”, only agents as the roots of spacetime. Anything else is just a state: information. So matter would be a state of spacetime — sort of like quantum field theory!
Semantics and dynamics
So much for the physics inspiration. From the Computer Science side of things, I had learned an appreciation of behavioural semantics. Although the term has its origins in linguistics, semantics of computer science are not about the meanings of words, but rather about the patterns of causal unfolding of processes: the protocols and interaction rules that convey information and behavioural patterns. Behavioural semantics refer to a kind of algebra of behavioural transitions. Such things are usually invariants in physics, so we don’t emphasize them. For example: the fact that charges repel or attract, how many kinds of charge there are and what happens when they meet, etc. The fact that entropy tends to increase on a certain scale. These might be called behavioural semantics in computing, but in physics they can’t be changed so there is no reason to give them a name. In Computer Science, the possibilities are far richer, and pretty much anything can be changed. It’s more like biology, where cells have many more degrees of freedom than elementary objects so that the domain of possibility is much larger. Indeed, I had already spent some years looking at immunology and this was definitely in mind when thinking about agents.
The emphasis on semantics in computing is natural because one is trying to engineer outcomes using agent characteristics as stepping stones. Let’s call those characteristics promises and say that they are autonomously decided. In physics, we treat these as just passive facts and one moves quickly to talk about the rates at which phenomena happen and the proportionalities with related phenomena. But there is a twist. In order to fully respect autonomy, it cannot be true that what one agent promises would necessarily be accepted by another. Promises might be offered autonomously (polarity +) but those behaviours might not be accepted (polarity -) by another. In computer science, one rarely gets beyond this kind of semantic design. As a result, computer science is terrible at scaling, because students don’t learn those concepts, or even differential equations, they learn about sets and linear algebra (if they learn any mathematics at all).
A goal of Promise Theory, then, was to unify these two issues more generally–starting from a kind of atomic theory of systems based on strictly autonomous agents. Very quickly, PT looked like molecular chemistry more than fundamental physics. After all, agents are most interesting when they are different. Physics deals with bulk properties that are homogeneous, but reactive chemistry is about interactions between agents with different semantics. There was still room for understanding bulk computers in a network, but these are not the problems computer science finds interesting. I have worked on some of these in my book: A Treatise on Systems, Vol II.
So, Promise Theory didn’t start out trying to calculate a specific experimental result, or explain one particular phenomenon. I used it to explain self-regulating systems that kept certain promises, which is an engineering problem. For a physicist, this is a bit too open-ended to be the kind of theory we are used to. On the other hand, quantum mechanics ends up being like that. It just had the benefit of having experimental phenomena with simple measurable semantics.
The formulation of Promise Theory has to begin with set valued quantities–from sets, all arithmetic stems via rings and fields, but in physics we cut to the continuous functions straight away and build a world around numbers. But, building on such elementary considerations as sets, like computing or biology, it takes quite a lot of work to reach a level at which one might discuss rates of change in a classic physics sense. Physics tends to simply postulate some rate equation (or gradient field) and start from there. I don’t want to get into the philosophy of how that happens, but we have to see that for what it is: it sweeps a great many assumptions under the theoretical rug, which are precisely the things we are questioning about agents. In particular physics relies almost exclusively on the continuum approximation–even to the extent of believing that the continuum is probably real (while also admitting that it probably isn’t!). Even attempts to model the smallest objects (string theory, etc) embed these discrete elements within continuous spaces and use rate equations to model change. There are good reasons for this, but now try to describe DNA synthesis using a differential equation of motion.
Hidden dimensions
One question that Promise Theory poses for physics is: what if spacetime is not a passive continuum, i.e. a mere theatrical backdrop for matter, replete with coordinates and simply connected dimensions, but is actually a collection of agents like a computer network or a biological tissue, whose dimensionality and sense of direction is only a statistical concept? This idea has been considered before by Myheim and later by Sorkin in “causal set theory”. But they didn’t consider spacetime to be active rather than passive (if we ignore spin, which remains an anomaly), nor did they really consider how geometry works or concepts like “uniform motion in a straight line”. In Promise Theory, instead of being passive and formless, every “point” is in fact an agent with interior degrees of freedom. How do we then describe things, and the motion of things through such a space.
Suggesting interior degrees of freedom at each point isn’t all that different from introducing the notion of extra dimensions in Kaluza-Klein or String Theory. These are also “hidden inside points” by suggesting they are so small as to be unresolvable for all intents and purposes. Only the geometric outer product of spaces requires them to be orthogonal dimensions–but that’s a question of representation not of physics. All such models that have additional “semantics” require processes that aren’t visible in the classical 3+1 dimensional spacetime. The main difference is that Kaluza-Klein theories rely on continuous geometry to model behaviours, whereas agents don’t need to make that assumption. Again, the assumption of continuity is more by habit than intent. The continuous dimensions often lead to the presence of too many symmetries for the phenomena one is trying to generate as in the Calabi-Yau issue of strings.
So how would one make something like spacetime out of agents? The semantics of spacetime in physics are the semantics of manifolds: open balls sewn together, with dimensions prescribed up front. For agents, forming graphs, we need to define what it means for one point to be next to another and how many dimensions make sense. I looked at this question in a few papers (more a series of notes) some years ago. It involved developing a more general notion of motion than we have in physics. Because, if you can’t measure distance and time without referring to agents, it’s not as simple as writing down a differential equation. In fact there are 3 different ways of defining motion–and these are actually realized in biology. I called them motion of the 1st, 2nd, and 3rd kinds. The notes are available for the curious, and I later summarized them in a popular book called Smart Spacetime
How Information Challenges our Ideas About Space, Time, and Process.
- Spacetimes with Semantics
- Spacetimes with Semantics (II), Scaling of agency, semantics, and tenancy
- Spacetimes with Semantics (III) — The Structure of Functional Knowledge Representation and Artificial Reasoning
- On the scaling of functional spaces, from smart cities to cloud computing
- Virtual Motion I: Motion of the 3rd Kind — Draft paper on virtual motion (2021)
- Virtual motion II: Notes on kinematics, dynamics, and relativity in Semantic Spacetime
- Virtual motion III: Motion of the Third Kind (III) On the Semantics of Motion in Discrete Spacetimes and Evolutionary Selection
The concrete outcome of this work was to inspire a way of representing knowledge is what we would now call Artificial Intelligence, but (in my own mind) it’s bigger than that.
Given that every theory we have builds on the enormously useful toolkit of Euclidean or Riemannian vector spaces and geometry, it sounds initially a bit stupid to deviate from that. But, from the viewpoint of parsimony, it’s quite natural. Why introduce a lot of points that can’t actually be observed, points that may not play any role and add concepts that might not exist? It’s a bit like Heisenberg rejecting anything except observables for quantum theory.
In another idea, while developing an idea for computer networks, I showed how entanglement could be represented by the famously impossible “hidden variables” (different from Bell’s definition) to yield the exact quantum behaviour of the Aspect experiment and sketch out the possible explanations for quantum behaviour from simple deterministic agents.
- Spacetime-Entangled Networks (I) Relativity and Observability of Stepwise Consensus
- Spacetime Entangled Networks (II) Bell characteristics and quantum similitude of active agent processes
- Entanglement of Software Agents: How some quantum weirdness may be explained by a software model
Now this is sounding speculative and it’s even verging into classic crank territory, but this is all straightforward application of agent semantics. Those who have grown up believing in the cult that quantum mechanics interpretation has become will immediately reject this. I wasn’t even allowed to put the second paper on the arxiv because it wasn’t considered publishable.
To some extent, I think physics has fallen into a self-defeating belief system that assumes algebraic generating methods (like the action principle, quantum fields, etc) are actually direct representations of reality rather than methods of calculation. Is the wavefunction real? Is it really a function of spacetime? What does that mean? The question has stunted progress in quantum mechanics for a century and separates theory into ideologies, crossing the line between impartial and whimsical in a quite unscientific way. For PT to connect with quantum theory, one has to ask: what are the agents? It’s tempting to fall into the trap of thinking that the agents are the particles floating around in space, but no! They are space itself, so particles and their properties would be entirely virtually promised attributes moving around. This aligns with Schwinger’s view that each volume element has a field value. The connections between the points also need to be defined. There is a whole can of worms to be understood, so only a fool would rush in and suggest rewriting the theory in terms of PT. But that doesn’t mean one can’t learn from it, by trying to answer exactly those questions.
Other issues arise when looking at agent mechanics. For example, conservation laws are extremely difficult to understand from an agent viewpoint. Even motion in a straight line is hard to justify. Physics takes such things as axioms and vector spaces are designed to make them so rather than to explain them. There seems to be no obvious reason why any property would be conserved. In physics, this is merely a tautological assumption. Noether’s theorem famously explains this, but actually it doesn’t. It simply says that the famously assumed continuity of spacetime is equivalent to the famously assumed conservation of energy and momentum, through a continuous generating function (essentially on dimensional grounds). Let’s not even get into what momentum and linear motion actually are (I do discuss it in my popular book Smart Spacetime).
As I’ve chipped away at issues in Promise Theory (which doesn’t pay bills), mostly limited by having to earn a living, I’ve tried to stick to the things that can be said clearly, rather than speculate about what I would like to see. This is the job, right? Not everyone plays it honestly, but we try not to fool ourselves. Happily, PT hasn’t disappointed, often coming up with unexpected surprises that were obvious with hindsight.
Across the scales
What I find remarkable about Promise Theory, not that I’m necessarily trying to advocate for it, is that (precisely by being general) it has something to say about processes at every scale. This too could be a red flag to physicists, who generally only think of one thing at a time and one scale at a time. That’s because there is always some causal language at every scale, about which one can formulate semantics. The details change, but the universality doesn’t.
For example, the basic causal semantics led to a formal definition of a few previously “moral” notions like authority, trust, cooperation, common knowledge, and a number of other social phenomena. While working on a study of data about trust in Wikipedia, I stumbled upon an explanation for Robin Dunbar’s famous group number hierarchy based on Promise Theory reasoning.
It’s hard to say that it’s a consequence specifically of Promise Theory, as one could maybe have figured it out by some other route, but without it it might not have been natural to put together that particular story… I don't know. That's just how I did it.
A final point is that Promise Theory predicts that agent behaviours on any scale can never be deterministic, except on average. This follows from insisting that agent interactions satisfy the constraints of Information Theory and the Nyquist-Shannon theorem. This is potentially an explanation for why quantum mechanics is non-deterministic when its key equations are deterministic. I find it remarkable that this alone doesn’t make physicists more interested in the arguments.
TL:DR Blah, blah, blah …
If all this sounds like a lot, I can confirm that there was a lot of work involved in figuring out these details, and yet it still all feels quite inadequate. Twenty years have passed quickly, but nothing about Promise Theory moves it past the status of being “merely promising”. It’s still probably “not even wrong”, like so many other theoretical ideas–even those that follow more of the forms of conservative physics.
When I hear negative remarks, I feel like rushing to patch someone’s perceived hole in understanding. But the pot shots are usually just intellectual laziness: people will always shoot at wantonly as if Promise Theory were nothing more than a symptom of unwelcome climate change. People will always ignore or even disparage what doesn’t appeal to them not because they necessarily object but because it’s what people expect of them. But this brings up mixed feelings. Promise Theory has shown its value in technology, but why wouldn’t other sciences adopt it? It would be so much more convincing if others could be persuaded to work on it too. Of course, I’m not much of a salesman. Why is this my job?
Alas, the skill set involved to make progress in something so technical is not at all common, even if someone could be convinced. It’s not about solving difficult equations. You have to be able to construct the ideas in a range of representations–which is a lot harder, because it can’t be taught. I’m often reminded of Machiavelli and his advice to The Prince:
“There is nothing more difficult to take in hand, more perilous to conduct or more uncertain of its success than to take the lead in the introduction of a new order of things, because the innovator has for enemies all those who have done well under the old scheme and lukewarm defenders in those who may do well under the new…”
All I can do personally is keep trying to do honest work, chip by chip, piece by piece, and hope that others will see something in these ideas too. PT isn’t the kind of theory you can write down in a single equation, so it will never appeal to everyone. Anyone who expects all theories to be like that is a bit naive, in my view. Still, I hope for physicists above all, because I do believe that only physicists can approach the range of skills needed to understand systems of agents and eventually reveal their implications.
So, come on people, how hard could it be?
You can read more about Promise Theory here: https://markburgess.org/promises.html
