Reciprocating Transformations between a Working Memory Updating Function and Multiple Imagery Generation Systems

Dr. Jared Edward Reser

Abstract

This article delineates a neurological process that is suggested to give rise to mental continuity. A loop is described where reciprocal interactions between bottom-up sensory areas and top-down association areas maintain, discard, and transform information resulting in the orchestration of imagery generation and internally guided thought. Nodes in posterior sensory areas are capable of recognizing and representing unimodal sensory features and patterns. Together, they are capable of combining individual features into composite, topographical maps or images. These topographical maps can be used to depict bottom-up environmental stimuli as well as internally driven top-down activity. Nodes in anterior association areas are multimodal and have a capacity for sustained activity, allowing the maintenance of pertinent, high-level features through elapsing time. The features are temporarily maintained in association areas and utilized as imagery specifications that are fed back into lower-order sensory areas where they are continually used in the construction of successive topographic maps. Salient features from these transient, sensory maps progress up the cortical hierarchy where they activate the corresponding nodes in association cortex, adding them to the store of temporarily maintained features. Thus, the most salient, novel, or goal-relevant features from the last several mappings are maintained in association areas. Gradual changes in this limited-capacity store of simultaneously coactivated association nodes occur as: 1) the nodes that continue to receive sufficient activation energy are maintained; 2) the nodes that receive reduced energy are released from activation; 3) new nodes that are tuned so as to receive sufficient energy from the current constellation of coactivates are converged upon, and incorporated into the remaining pool of active nodes from the previous cycle. This updated store of nodes is then used to construct the next sensory image. The fact that some nodes within association areas remain active for prolonged periods over the duration of several reciprocal top-down to bottom-up transformations is taken to account for the continuity found between successive topographic maps. The sustained and dynamically overlapping activity of higher-order association nodes allows consecutive topographic maps to have related content, exhibit progressive qualities, implement learned algorithms, and carry thematic or narrative continuity over sequential processing states. Included is a discussion of the implications that the model may have for defining intelligence, understanding consciousness, systemizing working memory, and constructing AI architectures.

Keywords: association cortex, cortical assembly, perception, primary sensory cortex, systems neuroscience, top-down

Introduction

Some important questions in cognitive neuroscience today include: 1) How can the process of thought, the sensations of consciousness, and the functionality of working memory be described in terms of brain events? 2) How do elemental features or fragments of long-term memory recombine to represent novel concepts and episodes? 3) What neurological events take place when mammals transition between brain states? 4) What is the nature of information transfer between association and sensory cortices? 5) How does the human brain permit higher capacity working memory functions relative to other animals? 6) What processes in the brain give rise to the mental continuity that humans experience? 7) Is it possible to reduce explicit, conscious processes down to their constituent, implicit, unconscious ones? Without being able to tie together all of the neurological, psychological, and philosophical loose ends necessary to answer these questions comprehensively, this paper will attempt to address them in an exploratory way using a novel approach. Rather than being based on traditional topics, this article will view these questions from the perspective of “mental continuity.” Continuity is defined as being uninterrupted in time. As proposed here, mental continuity involves a process where certain representational content exhibits uninterrupted persistence over time whereas other content is discarded or discontinued. This psychological process, and the thematic and narrative quality that it creates during internally generated thought, is made possible by sustained neural activity.

There are currently many illustrative and biologically plausible theories that address the questions listed above. Some do an exemplary job of tying together a large number of relevant phenomena into a cohesive picture. Models such as Baar’s global workspace theory (Baars, 1997; 2002), Baddeley’s model of working memory (Baddeley, 2000; 2007), Damasio’s convergence-divergence paradigm (Damasio, 1989; Meyer &Damasio, 2009), Edelman’s concepts of reentrance and neural Darwinism (Edelman, 1987; 2006), Edelman and Tononi’s conceptualization of a “functional cluster” or “dynamic core” (2001), Fuster’s conception of cognits (Fuster, 2009),  Tononi’s conception of integration of information (Tononi, 2004), and Grossberg and Carpenter’s adaptive resonance theory (Carpenter and Grossberg, 2003) have done much to lend perspective and insight into the mechanics of perception, attention, working memory and consciousness. In fact, concepts from these works (and several others that will be cited) form a battery of implicit assumptions about cognitive neuroscience that provides theoretical scaffolding for what is written here. Despite much progress, most scientists report that current theory is unsatisfying because it cannot yet bridge the gap between neural connectionism and subjective conscious experience (Chalmers, 1995; Chalmers, 2010; Shear, 1997). Further, even though many contemporary models largely agree with empirical data, little has been done to reconcile their disparate, piecemeal approaches (Pereira & Ricke, 2009; Vimal, 2009). Exploring the concept of mental continuity from both psychological and neurological perspectives may help integrate observations from cognitive science with those from neuroscience.

Animals are information processing agents. They receive information through sensory receptors, expose it to an existing network of information processing channels, and allow it to determine the pattern of motor output predicted to best respond to the situation at hand. When mammals do this, they match perceptual stimuli with preexisting, invariant, template-like representations held in memory. These representations are composed of groups of highly connected cortical neurons that can be repeatedly used to represent the most frequently concomitant, acontextual features of a common, reoccuring event or stimulus. Contextual or episodic features are not included in these semantic cortical memories. Mammals are capable of holding a number of such mnemonic representations active and using them to make predictions by allowing them to spread their activation energy together, throughout the cortical network, to converge on other representations that are the most closely associated with the current mix of representations. This allows mammals to remember or recall concepts that are not present in the immediate environment. Sustained firing of the neurons that embody the representation, allow it to be maintained over multiple perception-action cycles, in order to inform related sequences of behaviors. That a number of such representations can be conserved through multiple points in time lends continuity to mentation.

The general intention of the present article is to delineate a multistep neurological process suggested to be responsible for mental continuity. We consider how sustained neural firing of nodes in association cortex underlies the uninterrupted persistence of goal-relevant fragments of long term memory (LTM), allowing an interrelated sequence of topographic mental images in sensory areas. The framework encompasses several phenomena related to cognitive neuroscience and eventually culminates in an analogy involving the ambulatory behavior of an octopus whose pattern of locomotion is taken to resemble the pattern of cortical activation and deactivation. This work intends to use these concepts to integrate current theoretical approaches while attempting to remain consistent with prevailing knowledge.

Mental Continuity and Sustained Activity in the PFC and Association Areas

The human prefrontal cortex (PFC) is thought to be instrumental in cognitive control and to have the ability to orchestrate thought and action in accordance with internal goals. In the mammalian brain, prolonged firing of neurons in association areas, especially prefrontal and parietal areas, allows for the maintenance of specific features, patterns, and goals (Baddeley, 2007). Cognitive control is widely thought to stem from the active maintenance of patterns of activity in the PFC that represent goal-relevant features (Goldman-Rakic, 1995). The temporary persistence of these patterns ensures that they continue to transmit their effects on network weights as long as they remain active, biasing other processing, and affecting the interpretation of subsequent stimuli that occur during their episode of continual firing. This ensures that context is taken into account during action selection (Miller & Cohen, 2001). In contrast, neurons in other brain areas, including cortical sensory areas, remain active only for milliseconds unless sustained PFC input makes their continued activity possible (Fuster, 2009). The mammalian brain, and especially the human prefrontal cortex, has neurons that are capable of “sustained firing,” allowing them to generate action potentials at elevated rates for several seconds at a time (generally 1-30 seconds) (Fuster, 2009). A neuron may exhibit sustained firing due to: temporary changes in the strength of certain synapses (short term synaptic modification), its intrinsic biophysical properties, extrinsic circuit properties (reverberatory circuits), or dopaminergic innervation.

Because activity in PFC cells can be sustained, and does not fade away before the next instantiation of activity, there is a temporally dynamic and overlapping pattern of neural activity that makes possible the juggling of information in working memory (the system that actively holds multiple pieces of transitory information in mind, where they can be manipulated) (Reser, 2011). The human PFC is equivalent to a limited capacity store or buffer of goal-relevant representations that are constantly in flux as new representations are continually being added, some are being removed while others are being retained. The pattern of activity in the brain is constantly changing, but because some individual neurons persist during these changes, particular features of the overall pattern will be continuous, uninterrupted, or conserved over time. This distinct pattern of activity may be partly responsible for the maintenance of psychological continuity across sequential processing states (Reser, 2011, 2012, 2013).

The most enduringly active PFC nodes correspond to what the individual is most focused on, the underlying theme or element that stays the same as other contextual features fluctuate. We picture one scenario in our mind’s eye and this can often morph into a related but distinctly different scenario. Our brain is continuously keeping some elements online whether they are representations of things that are concrete and tangible or abstract and conjunctive. In other words, the distribution of active neurons in the brain transfigures gradually and incrementally from one configuration to another, instead of changing all at once.  The neurons that show persistent firing over a period of time do not all start and stop together. Rather, they often persist at different intervals, where the beginning of the activity of one neuron will actually coincide with the tails of others. The sustained activity of prioritized features in the brain is staggered and overlapping, ensuring that human thought features a continuous cascade of multiple cognitive elements that persist through time. If it were not for the phenomenon of sustained firing, instantaneous mental states would be discrete and isolated rather than continuous. Moreover, because information could not be carried over to subsequent states, the ability to process or make associations between temporally distant stimuli would be disrupted. This is why the prefrontal cortex is associated with working memory, executive function, mental modeling, planning, and goal setting. This may explain why agents without the analogue of a PFC, such as most life forms and current artificial intelligence, do not exhibit human-like higher-order thinking.

Human PFCs are larger, with more intrinsic and extrinsic connections compared to those of most other mammals, and this allows us to take more information, further through time before it is displaced. This allows us to better remember our recent thoughts and allows previous processes to better inform subsequent ones. This might influence us to assume that somehow thoughts are “longer” in humans than they are in other animals. However, if thought has an architectural geometry marked by length, then does it also have starting and stopping points? Perhaps there is no objective stopping or starting point of thought. Instead, thought itself may be composed of the startings and stoppings of huge numbers of individual elements that, when combined, create a dynamic and continuous whole (Reser, 2012, 2013). Again, it is not that human thoughts are longer than they are in other animals; instead, human thoughts are composed of larger sets of elements that are capable of remaining coactivated longer.

PFC neurons are tuned throughout life to best determine what aspects of the environment should be maintained, or released from maintenance, given the current scenario and its preceding circumstances. When confronted with a complex grouping of stimuli its intrinsic connections probably allow it to discern which stimuli should be temporarily maintained based on prior probability. This expertise is garnered slowly after particular constellations of neurons exhibiting sustained firing are rewarded for their role in mediating task proficiency. This process is perhaps best and most commonly exemplified by the ability to select and maintain strategically important representations from a forthcoming sentence (spoken or written). The sentence will be comprehended if: 1) the relevant representations are identified and maintained; 2) all of the necessary representations are maintained throughout the duration of the sentence; 3) the imagery system has enough experience with this group of representations to build the appropriate image, depicting them in the way they were intended to be depicted. Everyone has had the experience where either the wrong representations were anchored upon or the right representations could not be maintained for long enough and the sentence had to be repeated or reread.

In the most intelligent animals, motor output and sensory activity reflect several seconds of overlapping association activity. In a mouse, on the other hand, the motor and sensory output is informed by a much briefer window of uninterrupted activity. In animals lower on the phylogenetic scale, the proportion of nodes that are conserved over a one-second time interval is smaller on average.  Thus, one way to quantify mental continuity is to determine the percentage of previously active neural nodes that currently remain active. Highly intelligent mammals likely: 1) have a larger number of available nodes to select from, 2) coactivate a larger number of nodes simultaneously, and 3) have the capacity to prolong activation in goal-relevant association nodes for extended periods. The third feature augments associative searches by allowing specific features to be used as coactivates and serve as search function parameters for multiple cycles. The feature that we have referred to as mental continuity may be a major facet of the general factor of intelligence and may exhibit individual differences in humans where deficits in this capacity may map onto various clinical syndromes, such as psychosis, schizophrenia, mental retardation, cognitive aging, dementia, intoxication, prefrontal injury and others. Thus, fluid intelligence may derive from the number and duration of assemblies, whereas crystallized intelligence may derive from the connections between assemblies and their tuning properties. Nevertheless, why did this capacity for mental continuity, working memory, and fluid intelligence evolve, and what purpose does it serve? Mammals most likely evolved the capacity to sustain certain representations so that important groupings of representations could be interrogated and modeled.

Dopaminergic Contributions to Mental Continuity

The dopamine (DA) system exerts complex actions within several interrelated systems of the mammalian brain: motor function, motivation, reward attention, learning, and delayed response. Given its role in sustained activity, the mesocortical dopamine system, and the systems that it acts on, may be heavily involved in mental continuity. Dopamine sent from the ventral tegmental area (VTA) modulates the activity and timing of neural firing in the PFC, association cortices, and elsewhere. Dopamine neurotransmission in the PFC is thought to underlie the ability to internally represent, maintain, and update contextual information. Researchers have proposed that DA serves a “gating” function in the PFC, regulating the access of prioritized contextual representations into immediate or working memory (Baddeley & Hitch, 1994; Braver & Cohen, 1999). This gating function allows for the selection of task relevant information and non-routine actions in the face of interference from task irrelevant information and routine impulses. Information related to behavioral goals must be actively sustained such that these representations can bias behavior in favor of goal-directed activities over temporally extended periods (Miller & Cohen, 2001).

It has become clear that the activity of the DA/PFC system fluctuates with environmental demand (Fuster & Alexander, 1971). Many studies have suggested that the system is engaged when reward or punishment contingencies change. Both appetitive and aversive events have been shown to increase dopamine release in the VTA, causing sustained firing of PFC neurons (Seamans & Robbins, 2010). Seamans and Robbins (2010) elaborated a functional explanation to support this case. They have stated that when reward or punishment contingencies change, the DA system is phasically activated because it is adaptive for the animal to anchor upon and further process novel or unpredicted events. The PFC representations of rewarding, punishing, salient, uncertain, or unpredicted events are kept active over time to aid in the processing of their significance. Continued activation across time is necessary when the animal makes a prediction error, is uncertain, or feels pressured to better understand its situation. This form of increased cognitive effort is necessary to process the import of a novel event. Seamans and Robbins (2010) suggested that the DA/PFC system might play a major role in the way attentional resources are allocated in the effort to understand the meaning of patterns of stimuli and the strategies to cope with or take advantage of them.

It is important for mammals to identify and capture information about unexpected occurrences so that the processing taking place can systemize these occurrences in an attempt to identify systematic patterns. Because these features remain active and primed, they can be used repeatedly as specifications that guide the generation of apposite mental imagery in sensory areas (Reser, 2012). It is highly probable that sequences of lower-order topographic images depict and explore hypothetical relationships between the higher-order, top-down specifications. This amounts to a continual attempt to search sensory memory for a topographic image that can meaningfully incorporate the important features. It seems that reciprocating activity between the working memory updating system and the imagery generation system builds interrelated sequences of mental imagery that are used to form expectations and predictions. However, let us take a step backwards and ask, “What are the neural units that are maintained through time?” Simple, localized assemblies of cortical neurons are considered the building blocks of mental representations and the cornerstone of the framework so let us define them next.

Microscopic, Localized Assemblies Represent Fragments of LTM

The present model is consistent with connectionism and parallel distributed processing in that it conceptualizes mental representations as being built from interconnected networks of decentralized, semi-hierarchically organized, pattern-recognizing nodes that have multiple inputs and outputs (Gurney, 2009; Johnson-Laird, 1998). Like other biologically plausible neural network models, it envisions these nodes as microscopic, modular neural units in the cerebral cortex and assumes that each individual unit represents an elementary feature or stable “microrepresentation” of LTM (Meyer & Damasio, 2009). Like several other models of mind-brain processes (Cowan, 2005; Moscovich, 1992), this model views cognition as a system responsible for using representations in LTM to guide goal-directed processing. This involves locating and activating the most context-appropriate, preexisting representations in LTM. To construct a high-order LTM-based representation, the cortex must combine a subset of the numerous, fragmentary lower-order units at its disposal into an improvised, composite representation. Importantly, all mental representations are constrained in that they can only be built from combinations of preexisting units, each with their own unique receptive and projective fields. Thus, thought, consciousness, and working memory fundamentally involve the selection and intricate copresentation of lower-order fragments of LTM.

This fundamental, lower-order unit of cognition comprises a number of similarly-tuned neurons that are synaptically bound to create a functionally discrete assembly (Lansner, 2009). These have been called cognitive building blocks, “lego-like” elements, or “cognitive atoms” in the past. Because the neurons of such an assembly share highly similar receptive fields, they all respond to a particular aspect of the external environment, and they can be said to have a unique although primitive, “window on the world.” A single assembly then has an aspect of irreducibility in the sense that the majority of its constituent neurons usually fire together when they fire maximally. Evidence of such assemblies has been found (Lansner, 2009). These assemblies are capable of being activated to various degrees and capable of spreading their activation energy to assemblies to which they have been connected in a Hebbian fashion (Hebb, 1949). Assemblies are maximally activated when the simultaneous firing of multiple other assemblies with which they are associated converges upon them. Thus, the assemblies, like the neurons that compose them, function as “coincidence detectors” or “pattern recognition nodes” (Fujji et al., 1998). In general, when a neuron or assembly fires, the pattern that it represents is recognized. The cortical hierarchy observed from sensory to association cortex arises because simple patterns in sensory cortex converge upon second order assemblies to create a higher order representation. These assemblies converge upon others and so on, up toward nth-order assemblies in association cortex which are sensitive to the firing of very specific conglomerates of lower order assemblies. It is probable that lower-order assemblies in primary sensory cortex correspond to a particular facet of a stimulus, whereas higher-order assemblies correspond to a particular conjunction of stimuli.

These hypothetical assemblies may correspond to cortical minicolumns of cells. This is because minicolumns in sensory cortex have been demonstrated to consist of neurons with highly similar receptive fields and are thought to map onto a specific, elementary perceptual feature. The structure of the cerebral cortex is highly repetitive, considering that it is composed of millions of these nearly identical minicolumns (Lansner, 2009), each employing the same “cortical algorithm,” as thought by some neuroscientists. Exactly how these discrete fields of cells function and interact has been relatively mysterious since Vernon Mountcastle (1978) first postulated the columnar organization of the cerebral cortex. Minicolumns are composed of closely connected neural cell bodies that extend vertically and span the six layers of grey matter in the neocortex. There are supposedly around 20,000,000 minicolumns in the human cortex, each of which is about 30 to 40 micrometers in diameter comprising perhaps 80-120 neurons. Each minicolumn has its own inputs and outputs, and each performs neural computation to determine whether its inputs from other columns are sufficient to activate its outputs to other columns (Rochester et al., 1956). Most neurons in a column share the same receptive field, and even though they may play very different functional roles within the column, they each contribute to the column’s ability to encode a unitary feature (Moscovich et al., 2007).

This article will continue to refer to these building blocks as assemblies; however, it is meant to be implied that the cortical minicolumn is a likely candidate for this construct despite some reservations regarding its internal consistency and presumed unitary nature. Importantly, minicolumns contain anatomical and structural inconsistencies, making their boundaries fuzzy in numerous respects.  However, minicolumns are somewhat spatially distinct, contain neurons with highly qualitatively similar receptive fields, and contain the necessary communicative properties, as they span each of the cortical layers and communicate with other cortical assemblies (both near and far) as well as subcortical structures. Neurons may not constitute equally elegant candidates for these building blocks because, despite the fact that each neuron has a distinct and singular receptive field, their functional properties vary widely depending on their cell type and the layer in which they are found. Hypercolumns are also not good candidates, as they can be subdivided into subunits with various, qualitatively different receptive fields (Horton & Adams, 2005).

Cortical Assemblies Unite to Create Ensembles

It is unlikely that individual assemblies represent consciously perceptible constructs. In fact, if only one assembly was systematically removed from a complex mental representation, its absence could probably not be distinguished at verbal report. We will refer to coalitions of assemblies that represent a whole, consciously perceptible construct as an ensemble. This is a highly theoretical proposition but it will allow us to continue in our systematization. The ensemble is a helpful distinction that will lead us to conclude that when a psychologically perceptible construct is displaced from working memory (and the ensemble associated with it loses activation), each of the individual assemblies that constitute the ensemble would likely reduce their firing as well, unless the assembly has sources of activation independent of the deactivated ensemble (i.e., it belongs to more than one ensemble). In other words, assemblies (which are neural units) can be bound by experience to constitute an ensemble (which is a neuropsychological unit). Assemblies are bound together into ensembles in a Hebbian manner due to approximately simultaneous activation during experience. This is in some ways consistent with Joaquin Fuster’s concept of cognits – distributed memories or items of knowledge defined by patterns of connections between neuron populations associated through experience. Fuster (2009) emphasized that his cognits are hierarchically organized, link noncontiguous neurons, overlap, and interconnect profusely. It is quite unclear at this time how different aspects of our experience are parsed into distinct acontextual representational ensembles.

Assemblies are discrete and singular, whereas ensembles are fuzzy with boundaries that probably change each time they are activated. Assemblies correspond to specific, very primitive conjunctions and are required in great numbers to compose composite representations of complex, real-world objects and concepts. Ensembles are these composite representations and have variable, indefinite boundaries, as the experience of no two objects or concepts are exactly the same. Assemblies are preexisting, and they are found in microscopic, fixed locations. They are selected when activation energy passes through structurally descriptive hierarchical networks. Ensembles, on the other hand, may span these networks on a more macroscopic scale. They are mutable and inherently improvised. The characteristics of an ensemble can be reduced to the characters of its constituent assemblies just as a population can be reduced to the traits of its people.

Ensembles may or may not span large stretches of cortex. It may be better to think of ensembles as being localized within neuroanatomical modules where inferior temporal cortex (IT) holds ensembles for objects, parietal areas hold ensembles for spatial orientations, and prefrontal areas hold ensembles for propositional and time-delayed relationships. Such ensembles might be contained within a few millimeters of cortex. It may also be reasonable to expect the existence of larger module-spanning ensembles. It would be difficult to attempt to characterize ensembles as spanning the entire brain though because: 1) the part of the ensemble that took place in early sensory areas would be highly transient due to the absence of sustained firing in sensory areas, and 2) the percept of the concept would be unlikely to be reincarnated by the same neurons as sensory cortex is topographically oriented rather than categorically oriented. Instead, the term ensemble should probably be reserved for constellations of activity that are likely to be repurposed in a similar form due to the stability of their neural substrate.

I expect that ensembles are not static; instead, I expect them to be constantly reincarnated and transmuted as additional information is injected into them during the reciprocations between bottom-up and top-down areas. Because of this, delineating the borders of an ensemble on extended time scales is arbitrary and subjective. If the activity in the brain could be frozen in time, there would be a fixed and definite number of active assemblies, some more active than others. The instantaneous activity within association areas could be divided into a number of independent ensembles. Ensembles are momentary in the sense that they have a fixed form and fixed number of assemblies when frozen in time.  As time progresses though, these ensembles may be constantly reshaped by ongoing neural activity. In a sense, ensembles are continually fleshing out but never completely encompassing the higher-order concept that they embody. For instance, as a person thinks about a higher-order concept (such as a ball, or consciousness), only a particular subset of assemblies that are ordinarily statistically associated with this concept will be active. As their thoughts about the concept progress, this subset will change. Thus, the total set of neurons that are most closely probabilistically linked to a particular concept can be referred to as a “macroensemble.” The full complement of assemblies that constitute a macroensemble for a frequently used, high-order representation are probably never active all at once.

The Selection of New Assemblies and Ensembles: Polyassociativity

The way that neurons are selected for activity in this model is consistent with spreading activation theory. In spreading activation theory, associative networks can be searched by labeling a set of source nodes, which spread their activation energy to closely associated nodes. Nodes in these models traditionally correspond to knowledge and concept units, which may be congruent with both our conceptualization of assemblies and ensembles. The propagation of activation energy follows weighted links to other nodes. Several alternate paths through these links can reach the same end node. When enough of these alternate links reach the same node, this node is likely to be activated. In the brain, these links are thought to represent connections between neurons or assemblies and the weights are found in the synapses. When an assembly’s neurons are targeted by a bombardment of excitatory post synaptic potentials, this will lead to the “ignition” of the assembly, which happens when the cells within the input layers become excited enough to activate the pyramidal projection neurons associated with the assembly, causing it to fire out rapidly to the cells of other assemblies in the cortex. Ensembles may be similarly ignited when a sufficient number of their assemblies are targeted. When a new ensemble is converged upon it brings a new conceptual association into the stream of thought.

Unlike subcortical areas, strictly one-to-one, linear activation is probably rare in the cortex. In addition, unlike subcortical areas, information processing in the cortex is not compartmentalized into individual nuclei that are relatively isolated from processing occurring elsewhere. Rather, cortical information processing involves many-to-one (convergence) and one-to-many (divergence) interactions between a massively interconnected network of nodes.  A given assembly’s outputs will send postsynaptic potentials to both local and distant assemblies according to the pathways created during both early axonal migration and experientially determined connectivity. When an assembly receives sufficient activation energy from its inputs, it will fire at its targets (its projective field), often firing recurrently at the sources that targeted it (its receptive field) until the configuration of assemblies changes to the point where it no longer receives sufficient activation from the bottom-up or top-down assemblies that converge on it.

Cortical assemblies work cooperatively by spreading the activation energy necessary to recruit or converge upon the next set of assemblies that will be coactivated with the remaining assemblies from the previous cycle. Many authors favor the idea of “selfish,” rivalrous assemblies that compete with each other for activation energy. However, it may be just as appropriate to view them as patient, democratic assemblies that wait until enough of their peers signal them before they become active. An assembly is released from activation (deconvergence) when it no longer receives sufficient activation energy from its inputs. This may happen when the assembly’s relevance to the processing demands diminishes. An assembly may also be released from coactivation if a number of inhibitory neurons converge on it as when it becomes incompatible with processing priorities. This pooling of activity, which may act at the level of both assemblies and ensembles and may influence processing in both association and sensory areas will be referred to as “polyassociativity.”

Figure 1

The Characteristics of Polyassociativity:

Gradual additions to and subtractions from a pool of simultaneously coactivated neurons occur as:

1.      1) Assemblies that continue to receive sufficient activation energy from the network (or whose activity is sustained by other means, i.e. dopamine) are maintained over subsequent points in time.

2.      2)  Assemblies that receive sufficiently reduced energy are released from activation.

3.      3)  New assemblies, which are tuned to receive sufficient energy from the current constellation of coactivates, are converged upon, recruited, and incorporated into the remaining pool of active assemblies from the previous cycle.

4.      4)  Neurons, which are newly recruited, are those that have fired the most frequently in the past with the mix of currently active neurons. Thus what is capable of being maintained in working memory must be highly similar

5.      5)  Neurons, which are newly recruited may have fired with all of their current inputs in the past but always independently and never simultaneously until now.

The fifth point in Figure 1 above, necessitates the introduction of what we will call a “novel convergence event” (NCE). An NCE occurs when nodes B, C D and E have each repeatedly and persistently caused node F to fire in the past, although they have never fired all together to induce F to firing. It is generally thought that it is highly improbable for a brain to have the same instantaneous constellation of total activity twice during a lifetime, and this suggests that these NCEs must be very common. This amounts to a non-Hebbian form of convergence that may be prevalent in the brain, may have correlates in specific psychological phenomena, and may be responsible for neural selection at both levels of assemblies and ensembles. For instance, we may have never seen the Jeopardy prompt: “The name of a planet, an element, and a Roman god,” but each of the clues may contribute independently to unconscious neural convergence onto the ensemble representing the construct of “mercury.” By the same account, it is possible that NCEs might result in superfluous memories being recalled because if the brain has not had prior experience with a particular set of coactivates, it cannot know beforehand if the recalled memory will be applicable. However, it is likely that these NCEs are ordinarily beneficial for processing rather than confounding.

Importantly, the characteristics of polyassociativity demonstrate how a group of active representations may work together to select subsequent representations resulting in a self-perpetuating process that demonstrates continuity over time. See Figure 2 below. Outlining the process of polyassociativity in this way is meant to show that “computation” in the brain may be primarily directed at determining which neurons, assemblies, or ensembles should be brought into activation next. This concept of neural polyassociativity is considered to be scalable towards ensembles, in the sense that our next thought will be based on representations that are closely related to the mix of previously active representations. On millisecond timescales, our next thoughts are chosen for us based on how the currently active assemblies interact with the associative network.

FIG.2 is a diagram depicting “polyassociativity” and illustrating the ways in which high-level features are displaced, maintained, and newly activated in the brain to form a “stream” or “train” of thought. Each representation, which may correspond to a neural ensemble, is represented by a letter. 1) Shows that feature A has already been deactivated and that B, C, D and E are now coactivated. When coactivated, these features spread and pool their activation energy, resulting in the convergence of activity onto a new feature, F. Once F becomes active, it immediately becomes a coactivate, restarting the cycle. 2) Shows that feature B has been deactivated while C, D, E and F are coactivated and G is newly activated. 3) Shows that feature D but not C has been deactivated. In other words, what is deactivated is not necessarily what entered first, but what has proven to receive the least converging activity. C, E, F, and G coactivate and converge on H.

The longer the interval of time between two moments, the fewer the number of representations that will have been maintained. For instance, if the distance between time 1 and time 2 is 10 milliseconds, then a very large proportion of assemblies will be conserved over this period. If the time between time 1 and 2 is 10 seconds, then the proportion of assemblies at time 2, which have exhibited sustained and uninterrupted activity since time 1, will be much smaller. Imagine that Figure 2 depicts the mental content of a monkey that is playing a game that requires it to remember a signal over a delay period so that its response at time period three can be informed by knowledge of the signal. If the correct action requires the maintenance of memory C, the monkey will perform the task correctly, if it requires D then the visual and motor imagery that the monkey creates will not include a consideration for D, or the learned behavioral associations with it. On a related note, we can imagine a scenario where B, C, D, and E from step one of Figure 2 were immediately replaced by F, G, H, and I. Such processing system may still be using previous states to choose subsequent states. However, because no activity is sustained through time, there would be no continuity in such a system. There seems to be two obvious ways for natural variation to occur in such a system: to have variation in either the temporary storage capacity or the temporal duration of storage. Figure 2 features 4 units for the former and 4 time periods for the latter.

If the number of items able to be held in working memory increased to 5, and A, B, C, D, and E coactivated simultaneously, this could change the network dynamics in a way that node F, may no longer be the node that was maximally converged upon. Similarly, this could also be the case if the capacity were to shrink to 3 items rather than 4. Thus, increasing or decreasing the capacity may lead to different advantages and deficits. Decreasing the capacity may reduce working memory span but increase creative and spontaneous thought, facilitate reaction time, and increase attentiveness to the environment. On the other hand, increasing the capacity may increase working memory span and specificity of memory but necessitate a higher degree of prior network training to produce beneficial results. Thus, the large human capacity for working memory may owe in part to the increased network training made possible by mammalian mothering and the intergenerational resource flows exhibited by primates.

The brain uses a blind heuristic, summoning up the memory fragments that are the most statistically related to the currently active fragments. This keeps us from coactivating things that have not been highly related during past experience and ensures that the constellation of representations being maintained in mind must be highly similar to past constellations, and perhaps only allows small incremental variations. This means that we can only have new thoughts that we have already mostly had sometime in the past, and that it is difficult to think or do things that we haven’t mostly already thought or done before. This also means that the newest representational addition to thought may not be what we want it to be, and we may not understand why it has been selected. This limits creativity and the range of thought but ensures that memory will work in a reliable and stable way. In terms of information processing, this simple concept of polyassociativity may represent the brain’s hidden logic or computational algorithm.

I believe that this process of polyassociativity may work within the brain at a number of neural levels, such as the level of neurons, assemblies, and ensembles. I also believe that the process may take place within a number of different processing pathways and loops in the brain. This article however, will focus on its role at the level of ensembles during the loop between top-down and bottom-up processes. At this point, it is necessary that we build imagery and top-down to bottom-up reciprocations into the model. The following sections frame the thinking process as a succession of maps or images guided by a continually updating buffer of salient features. Importantly, this buffer receives a portion of its new content from the maps themselves.

Sensory Areas Generate Composite, Topographic Mappings of Representations Held in Association Areas

To a certain extent, sensory processing is known to be accomplished hierarchically. The visual system is a good example, and what follows is a simplified account. The primary visual area (V1) (along with the thalamus) processes the stream of information sent to it from the retina. Neurons here are concerned with distinguishing dots in a coordinate system. A row of active, adjacent neurons in the topographically arranged V1 may converge on neurons that are higher in the visual hierarchy. These higher neurons signal the inferred existence of rows of dots – the lines, edges, and curves that make up visual scenery. Even higher-order, “downstream” areas of extrastriate, secondary sensory cortex are tuned to recognize when these lines and curves come together to create more complex visual features amounting to the recognition of objects (inferior temporal cortex), scenes, and faces (fusiform face area), and the like. All active information in each of the lower sensory areas simultaneously spreads to higher order, multimodal, more globally communicative association areas, activating abstract representations.

Many pathways in the brain, such as the ventral visual pathway, appear to use a “structurally descriptive” architecture where multiple, low-level, nonaccidental features are allowed to converge together onto even more abstract, higher-order, generic, template-like features irrespective of topography. A structural description is defined as “a description of an object in terms of the nature of its constituent parts and the relationships between those parts (Wolfe et al., 2009).” This particular type of hierarchical processing is thought to allow perceptual invariance and robust postcategorical typology. Early visual cortex processes visual imagery and represents it pre-categorically in the sense that it does not hold any of the higher-order identities of the image or the affordances offered by it. For instance, early visual areas may perceive a keyboard, but it is not recognized as a keyboard there. The letters on the keys are not symbols there, only lines and curves. The keys are not recognized there as objects meant to be pressed. This type of lower-order imagery is processed by and appreciated in visual, motor, and associative brain areas higher in the processing hierarchy. These higher-order areas, in the same vein, process more complex and abstract elements of the keyboard (post-categorical memory) without redundantly holding what is held by the earlier visual areas. These abstracted, categorical representations are held in limited capacity in association areas, can be prolonged by rehearsal, and they imbue imagery with trait descriptives rather than topography descriptives.

Assemblies in lower-order sensory areas identify sensory features from the environment and combine them into composite mappings that mirror the geometric and topographic orientations present in the sensory input. The early visual system uses retinotopic maps that are organized with a geometry that is congruent to that used in the retina and the auditory system uses tonotopic maps where the mapping of stimuli is organized by tone frequency (Moscovich, 2007). Early sensory areas create topographic mappings from patterns recognized in the external environment, but may also combine top-down inputs from higher association cortex into composite, internally derived imagery. In his convergence-divergence framework, which may be mostly compatible with the present model, Damasio emphasized that during recall, association cortices send back divergent projections about the records needed to reconstruct original perceptual maps in early sensory cortices (Damasio & Meyer, 2009). Internally-derived imagery, such as that seen in the “mind’s eye” is probably also topographically organized because it is created by the same lower-order networks responsible for perceiving external stimuli. Thus, when we think and imagine, we activate and manipulate memories in early perceptual networks. The PFC and other associative areas do not appreciably influence processing in V1 or the LGN of the thalamus (the earliest of visual processing areas) via recurrent projections, but can profoundly influence the activity in extrastriate visual areas, which in turn influence V1. Thus, contemplative thought may take place on the same “Cartesian stage” on which the sensory experience takes place. Sense, remembered or reactivated, is the substrate of internally derived thinking. This conception is consistent with the “consolidation hypothesis,” which states that memory is stored in the same areas that allow active, real-time perception and function (Moscovitch et al., 2007). It is also consistent with the idea that remembering or imagining a particular sensory image largely activates the same neural networks that are involved in actually perceiving the imagery in the environment (Crick & Koch, 2003). The available population of assemblies in sensory cortex acts as an active canvas for either the environment (via feedforward connections) or expectation and imagination (via feedback connections). An important attribute of this system is that activity in lower order sensory areas can be quickly wiped clean to accommodate a new picture, whereas activity in higher-order areas lasts longer.

Early sensory cortex, composed of primary and secondary sensory areas, is known to have its own very short-term memory called “sensory memory.” Sensory memory has been shown to hold more information than does working memory, but it does so very transiently (echoic and iconic memory decay faster but hold a larger amount of precategorical information). Working memory is thought to hold between 4 (Cowan, 2005) and 7 (plus or minus 2) (Miller, 1956) chunks (such as numbers of letters), and it can retain these chunks if a person rehearses the material. Sensory memory, on the other hand, is capable of holding significantly more numbers and letters, although it does so very fleetingly (2.5 seconds for auditory sensory memory and 250 milliseconds for visual sensory memory), and it cannot be rehearsed. Attention to imagined visual imagery probably works in much the same way as attention to perceptual imagery, in the sense that the imagery holds more information than we can consciously attend to and that it fades within a quarter of a second if it is not bound to, or reactivated by,  higher-order associations.

The process of integrating and reconstituting diverse association specifications into sensory imagery is probably in many ways identical to the way that sensory areas combine features of the sensory environment to create early sensory perceptions. When sensory areas create perceptions based on inputs from the retina, they construct scenes by conjoining dissimilar elements into a unitary cohesive interpretation based on what they have been rewarded for creating in the past. Sensory areas must do the same thing with inputs, not from the retina but from the association areas, to create imagined imagery. This suggests that one can perceive the relationship between two abstract ideas only if one already has implicit information in the sensory cortex (and its hierarchical network of structural descriptions) about how to co-represent them in an image. If the person is missing instrumental conceptual knowledge in their sensory areas, then these areas will not be able to rapidly create the appropriate image and the person will not be able to “picture” or imagine the scene. The ability of sensory modules to use novel combinations of abstract representations and integrate them into a denovo, yet sensible mapping seems supernaturally mysterious but must stem from the incredibly informed “structurally descriptive” hierarchical networks involved. Future research efforts should focus on the ways in which higher-order representations are handed down a structurally descriptive hierarchy and portrayed in a coordinate field as conceptually integrated lower-order representations.

The cortical sensory system does what association areas cannot do on their own – take various components and rapidly and unconsciously integrate them into a visage that conforms to strict, environmentally-imposed constraints. The way in which sensory areas integrate when they construct images is informed by reality, as they have been tuned directly by real environmental inputs, unlike associative areas, which are tuned indirectly by reality due to the intervening effects of motivation, temporal delay and inference. In other words, the associative memory system selects appropriate associations that are reflective of what we have done and learned, whereas the maps from the sensory memory system reflect the stimulus configurations that we have been passively exposed to through the environment. During perception the bottom-up activity may be driving and the top-down may be modulatory; however, during imagination the top-down activity may be driving and the bottom-up may be modulatory.

Reciprocating Transformations Between Sensory and Association Cortex

The cortex is only one pattern recognizer high, however, information passes through its hierarchical system horizontally across the surface of the cortex. As you move up the neocortical hierarchy, from posterior sensory areas to anterior association areas, assemblies code for patterns that are more abstract. This is because higher-order assemblies have larger receptive fields, retain features from larger spatial areas, and involve longer stretches of time (Fuster, 2009). The brain’s connectivity allows reciprocating cross-talk between fleeting bottom-up imagery in early sensory cortex and lasting top-down activity in late association cortex and the PFC. These reciprocations allow humans to have progressive sequences of related thoughts, where the topographic mappings generated by lower-order sensory areas are guided by the representations that are held active in association areas (Reser 2011, 2012, 2013). See Figure 3 for clarification.

FIG 3. is a diagram depicting the reciprocal transformations of information between lower-order sensory nodes and higher-order PFC nodes. Sensory areas can only create one sensory image at a time, whereas the PFC is capable of holding the salient or goal-relevant features of several sequential images at the same time.

It is thought that object recognition, and perhaps other cognitive processes, involves two-way traffic of signal activity among various neural maps that stretch transversely through the cortex from early sensory areas to late association areas (Klimesch, Freunberger, & Sauseng, 2010). Bottom-up sensory areas deliver sensory information and top-down association areas deliver perceptual expectations in the form of templates or prototypes (Carpenter & Grossberg, 2003). According to Grossberg, as long as the difference between sensation and expectation does not exceed a set threshold called the “vigilance parameter,” the sensed object will be recognized as a member of the expected class. During both recognition and recall, these two systems may work together in reciprocal exchanges to determine category belongingness. The same processes may be involved in the polyassociative recall and recognition of related ensembles. Both forms of activity involve feedforward and feedback (reentrant) connections in the corticocortical and thalamocortical systems that bind topographic information from early sensory maps about the perceived object with higher-order information from later maps forming somewhat stable constellations of activity that can remain stable for tens or hundreds of milliseconds (Crick & Koch, 2003). Moreover, activity within these active networks tends to reciprocate on hierarchical pathways between sensory and associations areas, perhaps on the order of brain oscillations.

Crick and Koch (2003) advocated that a helpful way to consider this reciprocal activity is to imagine that the front of the brain is “looking at” the sensory systems in the back of the brain. This process is similar to watching a television program that can be controlled with ideas and conceptualizations. During perception, the predictive feedback from the template-like association ensembles may make incomplete or noisy perceptions in early sensory cortices more complete by retroactivating generic features normally associated with the stimulus and completing the expected pattern. The early visual areas constitute the “television” in this analogy because, unlike association areas, they portray fleeting information in the form of maps that are spatially isomorphic to the coordinates of the visual field.

Feedforward activation from bottom-up sensory areas selects among potential assemblies in association cortex (sometimes called the “fast feedforward sweep”). On the other hand, feedback activation from top-down association areas hands down specifications to early sensory cortex for use in imagery building. I expect that after the “fast feedforward sweep,” the PFC and association areas can be protagonized as saying the following, “Ok, we have identified the important features from the last internally generated image and combined them with the other features that we have been holding online from previous images. Let us engage in another round of imagery generation, this time with more emphasis on specific elements from the last image. It should be interesting to see how the visual system combines this updated set of higher-order features into a single, composite, lower-order, topographical map.” Recurrent connections and sustained firing make it possible for recent states to spill over into subsequent states, creating the context for them. In an abstract sense, each new topographic map is embedded in the previous one. This creates a cyclical, nested flow of information processing, which is depicted in Figure 4.

FIG. 4 is a diagram depicting the behavior of features that are held active in association areas. 1) Shows that the features B, C, D, and E, which are held active in the PFC, all spread their activation energy to lower-order sensory areas where a composite image is built that is based on prior experience with these features. 2) Shows that features involved in the retinotopic imagery from time sequence 1 converge on the PFC neurons responsible for feature F. Feature B drops out of activation, and C, D, E and F remain active and diverge back onto visual cortex. 3) Shows that the same process leads to G being activated and D being deactivated, mirroring the pattern of activity shown in Figure 2.

Thus, the cyclical oscillations of information between sensory and association areas allow them to learn from each other and to integrate their knowledge like two people in a conversation. The fact that they have fundamentally different perspectives on the world makes the “conversation” between them dynamic and informative for both because of the lack of redundancy. The crosstalk is similar to that between two specialists in related areas, interrogating each other about the nature of their common interests. Only one of these specialists keeps a list of previous topics on hand. The extended activation of assemblies in association areas changes the learning process in sensory areas as well. Prolonged activation causes synaptic changes throughout the cortex to reflect higher-order, temporally-structured representations, altering the weights of receptive fields, and tuning ensembles and their assemblies to be able to respond to even more temporally complex features in the future.

Sensory areas might use the specifications handed down from association areas to generate new imagery, but they must often elaborate on what they have been given with closely associated but unforeseeable embellishments. These unspecified, extemporized features built into early imagery probably provide much new content for the stream of thought. For example, if higher order nodes come to hold features supporting the representations for “pink,” “rabbit,” and “drum,” then the subsequent mappings in lower-order visual nodes may activate representations for a well-known battery advertisement, and the auditory nodes may activate the representation for the word “Energizer Bunny.” Thus, the concept of a battery might be introduced into the thought process in a polyassociative manner, despite the fact that the previous concepts alone had nothing to do with batteries. The mental imagery that is generated may constitute only a slight modification to the previous imagery or may be a paradigm shift away from it. The next configuration of concepts may seem wildly different because it evokes a different sensory image, but unless attention shifts dramatically, it is likely that many of the high-order representations remain in place even when the early sensory imagery changes profoundly. Hence, even when sequential images appear incompatible, they may be highly conceptually interrelated.

In reality, association areas have much more to converse with than simply a single retinotopic map. In fact, they feed their specifications to and receive specialized input from dozens of known topographic mapping areas. These areas are constantly responding to incoming activity in an attempt to pull up the most context-appropriate map in their repertoire. These sensory modules take specifications not only from association areas, but also from other sensory modules. Moreover, motor modules give and receive specifications while they are building their musculotopic imagery for movement. Thus, the PFC and other association areas direct progressive sequences of mental imagery in a number of topographic sensory and motor modules. Relative to Baddeley’s model of working memory (Baddeley, 2000; 2007), this relationship is congruent with the relationship of the central executive with the visuospatial sketchpad, the phonological (articulatory) loop, and the motor cortex.  This relationship may also be an integral mechanistic feature of Gazzaniga’s “interpreter.”The present model would be improved if it were expanded to include these other models in its description of aprocess of reciprocating transformations between a partially conserved store of multiple conceptual specifications and other nonconserved stores that integrate these specifications into veridical, topographic representations.

Association Areas May Have Their Own Brand of Nontopographic Imagery

Antonio Damasio proposed that early sensory cortices construct image space and that association cortices construct dispositional space that does not hold any imagery itself. I believe that association areas do hold imagery. They hold imagery of higher-order concepts that are disoriented from spatial mapping or retinotopic coordinates. The imagery created in association cortex embodies conceptual relationships and perceptually transcendent concerns. Association areas may hold true imagery in the sense that they can invoke high-level perceptions of things of which the person can become conscious. Thus, cortical areas involved in visual processing - from the posterior occipital pole to the anterior frontal pole – lie together on a continuum with coordinate bound maps on one side and abstract, conceptual imagery on the other. Further research into the deficits and intact abilities in patients with damaged sensory cortex may elucidate this issue. Consistent with Damasio; however, this model agrees that association areas do not possess all of the information held in the early sensory cortices that converge upon them. In other words, the firing of a grandmother neuron in the anterior temporal cortex alone does not produce a conscious visual depiction of a grandmother in the mind’s eye. One can probably not visualize a spatial, line-bound image of one’s grandmother without early visual cortex, just as they would not be able to visualize the keyboard discussed above.

What do imagery mappings in association areas such as the dlPFC look like? Early visual areas create retinotopic visual information because the inputs to the cortex correspond to the geometric arrays of photoreceptors in the retina. The dlPFC does not contain an objective input geometry that maps directly onto something real in the environment. Instead, the input, and thus the maps correspond to the placement and relative orientation of the lower-order projection inputs that were arranged during the evolution of the cortex. Thus, the question regarding the spatial architecture of higher order thought and its imagery can only be answered in the future by neurocartographic investigation of the unique connectional geometry found in higher-order areas.

I believe that the early visual cortex activation creates vibrant, experiential imagery simply because it has become correlated with the appearance of this imagery in the environment. Brain cells create a theatre of the mind because they have “taken on” certain external properties. If this is true, then imagery must be held everywhere because each part of the brain has become correlated with some type of environmentally induced experience. Like the neurons responsible for the sensations in a phantom limb, early visual neurons “hold” the experiential properties of experiences with which they have correlated in the past. Surely anterior association areas have also correlated with experiences, albeit abstract ones. Thus, purporting that association areas do not hold true imagery is like saying that imagery is held in the “dots” of primary visual cortex but not in the “contours” of secondary visual cortex. When you imagine things, from simple objects to abstract concepts, you experience them again because you fire the same neurons that fire when it was experienced. This thinking then frames consciousness as a jumbled up reflection of environmental occurrences. It is fascinating that we are able to appreciate a cohesive percept of a hodgepodge aggregate of previously distinct microrepresentations.

The idea that association areas may hold their own brand of abstractly mapped imagery frames the brain as a system of interacting modules specializing in mapping different topographies that are all trying to generate their best interpretation of what the other modules are doing. Because some of these modules have assemblies that fire for sustained periods, they are better positioned to direct activity through time. The dorsolateral PFC is a good example of a module with the capacity for sustained influence over modules specialized for visual and auditory processing, whereas an area such as the orbital PFC may direct maps generated by emotional and reward-related modules.

An Analogy Involving an Octopus:

The nature of the pattern of neural activation in the cortex is captured by an analogy, which involves a many-armed octopus grabbing and releasing footholds (ensembles made of cortical assemblies) as it pulls itself from place to place. The analogy captures several neurophysiological phenomena but also fails to capture others. It is meant to illustrate that the thought process involves the simultaneous coactivation of several ensembles at a time (multiple footholds held by an octopus) as well as the deactivation of previously active ensembles (the removal of an arm from a foothold) and the activation of previously inactive ensembles (the placement of an arm on a new foothold). This analogy may be valuable because it depicts a system, which even a child can understand, where specific nodes are conserved through time as others are actively repositioned.

In the present analogy, each octopus arm corresponds to an active ensemble, the suction cups on one arm can be taken to correspond to the assemblies that make up the ensemble while the grains of sand under each suction cup on an arm represent cortical neurons. This analogy may be apt because, like the grains of sand on the sea floor, cortical neurons do not move; rather, it is the pattern of activation – the octopus and its appendages - that move. The fact that the placement of some of its arms is conserved over sequential movements gives the octopus balance and stability, just as the conservation of some ensembles provides the physical basis for the continuity of thought.

I have developed and evaluated a few different models to represent the workings of the mind. While doing so, I realized that a good model would have to satisfy certain criteria. The original allegory that I used was of an ape swinging from branch to branch (hand over hand), where each branch represented a separate ensemble in the cortex that coded for a new representational addition to thought. This early analogy tried to convey the idea that the branches, or concepts, from the immediate past determine what branches would be held in the future. It was meant to convey that, during internally guided cognition, we move from one representation to the neurologically nearest, most appropriate representation in a mechanistic and deterministic manner. To me, the next chosen branch in the cortical canopy represented the probabilistically most likely association given the person’s current thought, past, and structure of their memory. This model of the thinking process is limited because it is linear. I came to understand that ensembles are not activated and then deactivated in linear sequence. This caricature of memory is limited and vague, and it failed to capture the polyassociative and unintermitting nature of thought.

Once I started to think nonlinearly, I concluded that mental activities must involve the simultaneous coactivation of numerous ensembles. It became clear that the next psychological association made was not dependent on a single neurological precursor, but rather several. I replaced the branch-swinging ape with a walking octopus. I changed the animal in the analogy, because the octopus has more arms, and it can simultaneously possess more footholds. The many arms introduced important and divergent features to the locomotive behavior that I think create instructive analogies when superimposed on the neural processes of thought. For example, the octopus analogy has the advantage of demonstrating how several interacting ensembles combine to drive the progression of thought. Moreover, because these ensembles remain active for different durations, thought does not stop and go in discrete steps but is continually “carried along” by the elements that endure through time. All ensembles and their neural assemblies will deactivate within a number of seconds, but the intermingling of ensembles of some temporal stability with those of more fleeting persistence sustains the associative bridges that allow the thematic and unifying consistency that is a hallmark of cognition.

The routine of ensemble activation and deactivation is very similar to “polypedal locomotion” or movement in animals with many legs. It is not much like the locomotion of an insect such as a millipede or a centipede though because these animals move their legs in stereotypical, repetitive ways where the placement of each leg is not actively influenced by the placements of other legs or of the qualities of the footholds. The pattern of activations in the brain is more like the polypedal locomotion of an octopus that is “seafloor walking” because it is asymmetrical, dynamic and the placement of the next legs is influenced by the octopus’ stance, posture, and the characteristics of the footholds themselves. Most importantly, this model can accommodate nonlinear aspects of neurodynamics. One ensemble does not activate the next in sequence. Several ensembles/assemblies are coactivated together polyassociatively, and they pool their activation energy to determine the ensembles/assemblies that will be activated next.

An Octopus Walking on a Cortical Sea Floor

In the present analogy, the octopus’ footholds represent simultaneously activated ensembles. However, just as thoughts do not hold still, the octopus is constantly repositioning its arms. The ensembles that are used are continually cycling as the “octopus” releases footholds in order to free up resources (arms) in order to grab new ones. Just as our working memory has a limited capacity, and is constantly forced to reallocate its resources, the octopus has a limited number of arms. Coincidentally, the number of chunks (psychologically perceptible units of perception and meaning) that can be held in working memory, 7 plus or minus 2 by some estimates, coincides with the number of arms that an octopus has (8).

FIG 5. Depicts an octopus within a brain in an attempt to communicate how mental continuity is made possible. When an octopus exhibits seafloor walking, it places most of its arms on the sand and gradually repositions arms in the direction of its movement. Similarly, the mental continuity exhibited in the brain is made possible because even though some representations are constantly being newly activated and others deactivated, a large number of representations remain active simultaneously. This process allows the persistence of “cognitive” content over elapsing time, and thus over information processing states.

Some ensembles can probably be retained even after the transitions between a number of thoughts. This happens when thoughts cycle and change but hold a common element or theme constant. When we attempt to solve a novel and complex problem, we try to keep the majority of our octopus arms firmly planted so that we can keep the problem set in mind. Some aspects of creative thinking or free association, on the other hand, might involve strategically pivoting around a smaller set of active ensembles and using these to determine the next set of coactivates.

It is not always the case that the majority of ensembles are conserved from one thought to another. When they become a lower priority, all ensembles can be dropped or abandoned at the same time. This readily happens when we are exposed to a new, salient, perhaps emotionally laden, stimulus. When this occurs, the octopus “jumps,” taking all of its arms with it, and reorients to the new stimulus and its accompanying set of features. Such a jump would constitute an interregnum or disruption of mental continuity. Disruptions in continuity might occur due to a distracting stimulus in the environment or a disparate but engaging internally generated stimulus. Evolution has probably programmed the octopus to jump and reposition its arms quickly in order to respond to important sensory stimuli, so that mammals react to them with all of their cognitive resources.

Mental continuity is less easily disrupted in humans than it is in other mammals, although perhaps more easily disrupted in people with habituation deficits. Attention and distraction must be intimately related to the temporal conservation of ensembles. In fact, the extent of attention deficit and distractibility should be inversely related to the neurological capacity to conserve representations in association areas from second to second. Creating an operational definition for this proportion and ways to measure it (perhaps on a scale of assemblies per millisecond) may prove informative and may represent a biological measure of general intelligence. Stretching the analogy further, we might say that octopus arms shift, altering the grains of sand encompassed by each suction cup, just as the constituent assemblies of an ensemble shift as (for example) an ensemble for a mug fleshes out the macroensemble for a cup. In some people this shifting may be more pronounced or more loosely regulated.

Another part of this analogy is the idea that the octopus will “topple” if it loses its grip on a sufficient number of ensembles. This makes the body of the octopus analogous to consciousness because brains become unconscious once coactivation (especially in the frontal and parietal fields) is sufficiently diminished. Assemblies in early sensory areas are often active during unconscious states, but assemblies in association areas are less active and out of sync with those in sensory areas. Thus, anterior-posterior balance and coordination are important for our allegorical octopus. Table 1 below summarizes some of the terminological concepts of this analogy. Neurological concepts correspond to particles or regions of sand whereas psychological concepts correspond to aspects or the behavior of the octopus.

Table 1: Definitions of Terms

Other Sources of Mental Continuity

This article appears to suggest that sustained firing related to the mesocortical dopamine system is fully responsible for mental continuity. Cortical priming and the hippocampus must play a role as well; however, it is probably relatively difficult to disentangle the similar influences that these three processes have on the progression of thought.  It is known that cortical nodes recently used in working memory maintain some activity, and due to this priming effect contribute to non-hippocampal dependent short-term memory. It is thought that these nodes remain temporarily metabolically active, and fire action potentials slightly above baseline. Even though these primed nodes take a background role, they are still much more likely than inactive nodes to be reactivated in the near future. Primed nodes thus must contribute to imagery and polyassociativity because of the way they continue to apply their activation energy to the cortical network. This implies that the footholds that our octopus recently released influence it heavily. It also suggests that disruptions in the continuity of highly active representations may be occurring constantly, but that it can be fairly easy for the train of thought to shift back to a previous track. Thus continuity may be less gradual than depicted in Figure 4, and more punctuated.

Hippocampal nodes make similar contributions to continuity using a different mechanism. Hippocampal dependent processes allow a different kind of continuity apart from the one we have been discussing, one that exists on a longer, yet fractured time scale. Even when a set of representations is completely deactivated and no longer used as an uninterrupted coactivate (a disruption in continuity), it can be brought back online (punctuated continuity), virtually in entirety, due to the intervention of the hippocampus. We can call this punctuated continuity, the recurrence of mental constructs. This recurrence may lend a consistency to consciousness in the same way that a motif (a distinctive and recurring form) gives a literary, artistic or musical work continuity.  The octopedal pattern of coactivation among represenations in the cortex allows continuity on the order of milliseconds to seconds, priming allows continuity on the order of minutes, but the long-term memory of the hippocampus allows mental continuity on the order of several seconds to years. The hippocampus, because of its connectivity with the neocortex, has the ability to detect the presence of a subset of assemblies that were coactivated together in the past and autoassociatively reactivate the rest of the assemblies that were previously coincident with the subset in a process called “pattern completion (McNaughton, 1991).” Hence, the hippocampus has the ability to guide the arms of the octopus toward historically coactivated footsteps that may have more to do with a specific scenario than motivationally relevant concerns. The hippocampus and PFC work synergistically though. The PFC keeps several things active long enough so that when the hippocampus takes a snap shot of an episodic pattern, the snap shot contains activates suspended from various moments in time.

The cortex without the hippocampus can model only semantic, categorical and sensory knowledge, but with the hippocampus, it can record individual instances of past sensory configurations to allow the creation of contextual and episodic knowledge. The formation of associations between representations in long-term memory within the cortex necessitates a number of learning cycles, where each instance of learning can only slightly strengthen or weaken existing connections. The hippocampus, on the other hand, is capable of associating discrete groups of representations from single episodic occurrences. These hippocampal coalitions, if routinely and habitually activated over time, can come to adjust cortical weights so that their content forms its own ensemble. Many of our ensembles probably arise from hippocampal dependent learning.

A second form of punctuated continuity may be found in neural coalitions that are bound by the VTA that represent key combinations of representations that have come to predict reward and thus are related to drive and motivation. Like hippocampal continuity, these patterns reoccur sporadically but are often not continuously active. The consistency of our wants, likes, goals, motivations and desires may constitute a form of punctuated continuity that is as responsible for conscious experience as any of the other forms. This may be true, in my opinion, because nothing makes us feel like us, more than our eccentric and eclectic set of reoccuring priorities and appetites.

Dual Processing, and the Unconscious

In psychology, dual process theory suggests that implicit and explicit processes use two fundamentally different cognitive processing systems, “system 1” and “system 2.” System 1 is implicit, automatic, and unconscious. It operates effortlessly and has a large capacity. System 2 is explicit, controlled, and conscious. It operates with effort and has a smaller capacity. These two systems are thought to be distinct and subserved by different general neural architectures, although they may actually use the same general architecture. This could be the case if the processing of system 2 is actually system 1 exhibiting neural continuity. That is, what is referred to as system 2 may simply be the processing architecture of system 1 continuing to process using the same general suite of representations for a prolonged period. Again, the system 1 is making automatic, intuitive flash judgments, but because of the mental continuity made possible by sustained firing, these rapid associations are able to support and buttress each other in a progressive and additive manner. For example, early processes may provide premises or propositional stances that can be used syllogistically to induce or justify a conclusion in subsequent processes.

System 2 may emerge when we have several system 1 thoughts that are all interrelated. Instead of jumping to a conclusion or judgment about something based on a single impulse, system 2 involves the sequential arrangement of several impulses that, progressively and cooperatively, lead to a higher-order decision or judgment. Multiplying two, two-digit numbers may be a mental calculation that involves a series of impulses coordinated together to implement a learned pattern of algorithmic steps. Most importantly, system 2 involves an extended series of system 1 operations, i.e., continuous operations with shared or interrelated content rather than disparate content. System 2 cognition may be present when several nodes in association areas exhibit sustained firing and are used multiple times to build topographic maps, culminating in sensory or motor imagery that could not be informed by any of the intermediate steps alone, or that is capable of solving a problem too difficult for any system 1 process itself. Thus, all instances of cognition may fall on a continuum between impulses that are short, discrete, and have little continuity to them, and series of interrelated impulses that demonstrate continuity, and are orchestrated by the conservation of several representations across time. This may be the essence of executive control. Researchers in psychology suggest that system 2 processes are initiated by motivation, surprise and curiosity. Of course motivation, surprise, and curiosity are the same factors responsible for recruiting the mesocortical dopamine system necessary to allow for the sustained PFC firing responsible for mental continuity. 

Is there a fundamental equivalency between conscious processes and unconscious ones? Can consciousness be reduced to an unbroken progression of gradually transforming unconscious impulses? Perhaps consciousness may simply be unconscious, associative processing with the continued activation of certain representations over successive instances of imagery generation. There is more to the unconscious though than simply these dead, automatic, zombie-like impulses. The unconscious can exhibit adaptive or intelligent properties. Such as when an automatic process is influenced by a single preceding event. This may happen when previous concurrent groups of ensembles fire together strengthening the bond between each of them making it so that only a subset of them are now capable of activating the ensembles that previously took all of them. With respect to figure 2, this may happen when B,C,D, and E become strongly bound so that in the future, only three of these might be sufficient to activate F. We are often unaware of the statistical regularity in our environment that underlies how currently active representations converge onto other representations. We often confabulate to explain this.

Implications for Artificial Intelligence

The present model may have implications for structuring artificial intelligence (AI). Typical AI systems are designed to perceive the environment, evaluate objects therein, select an action, act, and record the action, along with its efficacy and the results thereof to memory. There are no forms of artificial intelligence that do these things using mental continuity as described here. There are existing computing architectures with limited forms of continuity where the current state is a function of the previous state, and where data is entered into a limited capacity buffer to inform other processes. However, the memory buffer is not multimodal, not positioned at the top of a hierarchical system and does not interact with and inform topographic imagery. The model described here could be used to design a modular, hierarchically organized, artificial intelligence system that features a working memory updating function and the capacity for imagery generation. The prioritized features that are maintained over time could be used to create and guide the construction of topographic maps as well as the construction of natural language and the guidance of robotic movement.

It is currently not possible to engineer the human brain in a way that would increase the number and duration of active higher-order representations. However, in a biomimetic instantiation, it would be fairly easy to increase both the number and duration of simultaneously active higher-order representations. Accomplishing this would allow the imagery that is created to be informed by a larger number of concerns, and would ensure that important features were not omitted simply because their activity could not be sustained due to biological limitations. Of course, such a system would require extensive supervised and unsupervised learning.

Because cortical assemblies are essentially pattern recognition nodes organized in a hierarchical system, they should be able to be modeled by computers. The best way to do this with modern technology would be to use the artificial neural network architecture. An artificial neural network is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionistic approach to computation. Artificial neural networks are generally adaptive systems capable of complex global behavior, that alter their own structure based on the nonlinear processing of either external or internal information that flows through the network (Russel et al., 2003).

The program would have an embedded processing hierarchy composed of a highly interconnected multilayer neural network of pattern recognizing nodes that are organized into a hierarchical architecture similar to that of the mammalian neocortex. Like neural assemblies, the nodes exhibit a continuous gradient from low-order nodes that code for sensory features to high-order nodes that code for temporally or spatially extended relationships between such features. The lower order nodes are organized into modules by sensory modality. In each module, nodes work both competitively and cooperatively to create topographic maps. The network’s connectivity would allow reciprocating cross-talk between fleeting bottom-up imagery in sensory layers and a buffer of lasting, top-down specifications held in association layers.

Nodes are grouped according to the feature they are being trained to recognize. The architecture could feature the aspects of both Kohonen and Hopfield networks, back propagation, self-organizing maps, bidirectionality, Hebbian learning as well as a combination between principal-components learning and competitive learning. The agent discussed here could be capable of integrating multiple existing AI programs, that are specialized for specific tasks into a larger composite of coordinated systems.

Nodes lower in the hierarchy are trained to recognize and represent sensory features and are capable of combining individual features or patterns into metric topographical maps or images. Low-order nodes are unimodal, and organized by sensory modality (visual, auditory, etc.) into individual modules. Nodes high in the hierarchy are multimodal, module independent, and have a capacity for sustained activity, allowing the conservation of pertinent, high-level features through elapsing time. The higher nodes are integrated into the architecture in a way that makes them capable of identifying a plurality of goal-relevant features from both internal imagery and environmental input and temporarily maintaining these as a form prioritized information. The system is structured to allow repetitive, reciprocal interactions between the lower bottom-up and higher top-down nodes. The features that the higher nodes encode are utilized as inputs that are fed back into lower-order sensory nodes where they are continually used for the construction of successive topographic maps. The higher nodes select new features from each mapping to add to the store of temporarily maintained features. Thus the most salient or goal-relevant features from the last several mappings are maintained. The group of active, higher-order nodes is constantly updated, where some nodes are newly added, some are removed, yet a relatively large number is retained. This updated list is then used to construct the next sensory image which will be necessarily similar but not identical to the previous image. The differential, sustained activity of a subset of high-order nodes allows thematic continuity to persist over sequential processing states. If this sustained firing is programmed to happen at even longer intervals, and even larger numbers of nodes, the system would exhibit a higher capacity for continuity. This would increase the ability of the network to make associations between temporally distant stimuli and allow its actions to be informed by more temporally distant features and occurrences.

FIG 6: illustrates how relevant features can be maintained through time using nodes with sustained firing. The figure compares the number of past nodes that remain active at the present time (“now”) in a normal human, a human with PFC dysfunction, and the hypothetical AI agent. The AI agent is able to maintain a larger number of higher-order nodes though a longer time span, ensuring that its perceptions and actions now will be informed by a larger amount of recent information. Note how the lower-order sensory and motor features are the same in each graph with respect to their number and duration, yet those in association areas are the highest in both number and duration for agent C.

Conclusions

What fundamental processes allow thought to move through space and time, or in another word, to “propagate?” Such processes must be grounded in physics, biology and information processing because they must explain how the physical substrate of intelligence operates in a mechanistic sense (Chalmers, 2010). This article attempts to present a general model of how memory is retrieved and manipulated in the brain. It includes a discussion of the neural basis of mental representations and the spatial and temporal pattern of neural activity that gives rise to thought, working memory and other psychological phenomena. The writing here is exploratory, contains untested assumptions, and many important concerns were left out of the discussion. A more complete and refined version would focus on better integration of existing knowledge from functional neuroanatomy, multisensory integration, clinical neuropsychology, brain oscillations, binding and attention. This article should also look more closely for specific points of evidence to bolster the claims made. To a certain extent, however, the article introduces new concepts and uses valid convergent reasoning in the absence of pertinent data and existing literature.

Here, the smallest functional unit of memory in the brain, a cortical assembly, is suggested to be a small group of neurons (perhaps a minicolumn) tuned to code for a discrete element of long-term memory. Perceptible mental representations, termed ensembles, are suggested to be large groups of assemblies that have been bound due to simultaneous activity in the past. When multiple representations are coactivated the individual elements spread and pool their activation energy resulting in the selection of representations for deactivation, activation and sustained activation. This process, referred to as polyassociativity, is suggested to work on the level of assemblies and ensembles, is suggested to select the content for the stream of thought and in doing so, lead to novel convergence events. How this process influences top-down to bottom-up reciprocations is also described. Activity from active representations fluctuates back and forth between early, bottom-up sensory cortex (where activity is metric, topographic and transient) and late, top-down association cortex (where activity is abstract, conceptual and persistent) perhaps on the order of brain oscillations. Currently active representations in association areas spread their unique constellation of activity backwards through the structurally descriptive hierarchy towards sensory areas, where they are used to guide the construction of composite topographical maps. Sensory areas and association areas continually stimulate each other into building interpretations of the other’s outputs resulting in a conversational interchange with minimized informational redundancy.

The fact that some assemblies within association areas remain active for prolonged periods, over the duration of several reciprocal top-down to bottom-up communications, is taken to account for instances of continuity found between successive topographic maps. The longer assemblies in association areas can be continuously activated - over a series of states - the longer they can influence sequences of sensory imagery in a sustained and consistent way allowing continuous, progressive alterations to the imagery. This feature of continued activation augments associative searches by allowing specific features to be used as function parameters for more than one cycle. This process is ultimately responsible for modeling, planning and decision making in general. The result is a stream of consciousness where each thought is quantitatively different from preceding thoughts, as newly relevant assemblies are added and the least relevant ones are removed. This process is reviewed below in figure 7.

FIG. 7. A list of processes involved in the central mental continuity algorithm

1)            Either bottom-up sensory information from sense receptors (environmental perception), or top-down information from association nodes (internal guidance), or both are sent to low-order network layers in sensory cortex that contain feature extracting cells.

2)            A topographic sensory map is made by each low-order, sensory module, one for each of various sense modalities. These topographic maps represent the networks best attempt at integrating and reconciling the disparate feature specifications into a single composite, topographic depiction. The map that is created is based on prior probability and training experience with these features. In order to integrate the disparate features into a meaningful image, the map making neurons will usually be forced to introduce new features.

3)            Information from the imagery travels bottom-up through hierarchical, structurally informed, networks of neurons in a polyassociative manner. The salient or goal-relevant features that are introduced during imagery generation are extracted through a perceptual process where active, lower-order nodes spread their activity to higher-order nodes. This is sometimes called the fast feedforward scan.  As the new features pass through the neural network, some are given priority and are used to update the limited-capacity, working memory, storage buffer that is composed of active high-level nodes in association cortex.

4)            Salient features that cohere with features that are already active in the higher-order nodes are added to the already active features there. The association representations that are selected are also influenced by ongoing spreading activity arising from the association area itself.  The least relevant, least converged upon features in higher-order areas are dropped from activation. The sustained firing of a subset of higher-order nodes allows the important features of the last few maps to be maintained in an active state.

5)            At this point it is necessary for the system to implement a program that allows it to decide if it will continue operating on the previously held nodes or redirect its attention to the newly introduced nodes. Each time the new features garnered from the topographic maps are used to update the working memory store, the agent must decide what percentage of previously active higher-order nodes should be deactivated in order to reallocate processing resources to the newest set of salient features. Prior probability with saliency training will determine the extent to which previously active nodes will continue to remain active.

6)            The updated subset of higher-order nodes will then spread its activity backwards toward lower-order sensory nodes in order to activate a different set of low-order nodes culminating in different topographic sensory map.

7)            A. The process repeats.

B. Salient sensory information from the actual environment interrupts the process. The lower-order nodes and their imagery, as well as the higher-order nodes and their priorities, are refocused on the new incoming stimuli.

Brain activity in association areas was likened to the “polypedal locomotion” of an octopus that is “seafloor walking.” This octopus leaves the majority of its arms where they are, yet is constantly removing and replacing a minority of arms. Even though ensembles are constantly being deactivated, we take many ensembles with us through time. This process allows us to transition between closely-related thoughts, and may constitute the “fabric of experience.”  If we did not do this, we could not be informed of what we were just thinking, and we could not have an interrelated, and progressive train of thoughts. Similarly, the longer certain assemblies are activated, the more new thoughts are informed by recent thinking. An intelligent person, endowed with an advanced capability to prolong activation of certain assemblies, creates temporally compounded patterns of representational complexity that allows them to be discerning, astute and sapient.

Different species in the animal kingdom exhibit large variation in the ability to sustain specific representations. Many mammals have not evolved a human-like capacity for sustained firing in PFC neurons and thus may have a lower frequency of associations made between temporally distant stimuli. This may suggest that in some environments it is less helpful to search memory for relationships between stimuli that occur in delayed succession and instead to focus on those occurring in quick succession. The cognitive strategy of these animals emphasizes the training of bottom-up reactions rather than top-down modeling (Reser, 2007). This capacity appears relatively fixed and inflexible from species to species; however, phasic dopaminergic responses can vastly increase the ability to reach back into the immediate past. Perceived goal relevance is the key. Phasically increased dopaminergic transmission in the PFC ensures that the brain will dedicate time and processing resources to an attempt to systemize and model a specific set of contextual features for a prolonged period.

It is commonly pointed out that executive functions help to protect task relevant information from interference from newer, task-irrelevant information. This form of temporally sustained information may be less relevant in some animals and may have a tendency to interfere and distract bottom-up responses that are crucial for survival. Extended persistence of neural activity may cause animals to react slowly to their environment because their imagery is influenced by past instead of present concerns in real-time. These animals would probably find the prolonged activity of representations distracting, noisy, and task-irrelevant. Thus, in animals with a less cognitively demanding neuroecological niche, the PFC’s role in keeping the last few representations online for an extended duration is diminished in importance, and incoming information from the last several hundred milliseconds is given priority over information from the last few seconds. For each species, the question becomes: “how far back in the immediate past do I want to go for representational specifications for building my current imagery and my current action?” Thus mental continuity has evolutionary benefits that our species adapted to necessitate, but its costs may have been prohibitively maladaptive for other species - it slows the octopus down.

Cognitive neuroscientists are now asking: “where and when does consciousness happen?” Most neuroscientists agree that when becoming apprised of sensory information from the environment, it takes a quarter of a second to become aware of the information. This time is thought to correspond to the time it takes for lower sensory areas to activate the PFC. Once the PFC is notified, it is thought that we become conscious of the sensory stimulus. In some respects, this seems reasonable except the PFC does not contain topographic sensory imagery. Thus, perhaps we must wait for the PFC and other high-order areas to contact sensory areas in order to experience our PFC’s response to a stimulus. Then perhaps we must wait for the PFC’s reappraisal of this internally generated imagery. Clearly this could become an endless cycle.  Is it necessary to have multiple rounds of top-down to bottom-up interpretations in order to be conscious of something? Must consciousness necessitate that ensembles related to self-awareness are coactivated? Instead, can we simply say that any reportable representations involved in mental continuity are conscious? Here consciousness is taken to be an elaborative process, where the longer a group of representations is maintained over a series of cycles, the more conscious one is of it. The longer the cortical octopus holds two concepts in its embrace during alternating cycles of mental imagery, the more the relationship between those two concepts will become conscious. Importantly for learning theory, the more closely associated these two concepts come to be, the more their imagined aspects become chunked together and the more the association between them is likely to become implicit.

These questions lead us to formalizing an operational definition for mental continuity. It may be reasonable to posit that mental continuity necessitates a “full cycle” between bottom-up and top-down processing. In my opinion, a group of representation must travel from association cortex to sensory cortex and back (and/or perhaps vice versa, from sensory, to association and back) to constitute a complete individual thought marked by mental continuity. The methodology from Giulio Tononi’s Integrated Information Theory of consciousness may be applied here. Tononi suggests that conscious information is highly differentiated yet also highly integrated (2004) and offers a quantifiable measure of the integration of information. It may be possible to apply his mathematics towards the idea of the integration of information over periods of time, to propound on the present concept of mental continuity.

A quale (an individual instances of subjective conscious experience) may gain its experiential qualities when the features of the quale are maintained through time, volleyed between association and sensory areas, and used to uncover apposite memories and images. This kind of thinking may cause us to wonder whether the persistence of information between processing states is necessary for conscious thought and what form of consciousness or sentience an agent could possess if it had no mental continuity. Unconscious zombies of the type encountered in philosophical studies of the mind may necessarily not experience mental continuity. Computers certainly do not experience the same type of continuity that humans do. Programming the computational architecture for an artificially intelligent agent capable of performing human behaviors, in human-like ways, without using mental continuity may necessitate a battery of if-then rules and subsystems to coldly mimic the process whereas building it into a computer’s architecture may actually prove to be less complicated.

Crick and Koch have advocated that neuroscientists should concentrate on finding “neural correlates of consciousness,” defined as the smallest set of brain mechanisms and events sufficient for some specific phenomenal state. Mental continuity as described here may be an integral element of consciousness and may be a strong candidate for a “neural correlate of consciousness.” Philosophers and neuroscientists have identified many different elements of brain function (thalamocortical loops and reentrant cortical projections) and attempted to explain how these may lead to conscious experience. I think that the present concept of “continuity through differential temporal persistence of distributed representational activity” is instructive and I even feel that it may be a central aspect of conscious experience, intentionality and qualia. It may be an intrinsic property of our universe that matter may be organized into a form that can support mental continuity. Moreover, mental continuity may help offer a perspective to help strengthen the materialist view that that mental properties can be metaphysically reduced to physical properties. Is it possible that an individual’s identity and “self” may simply be this wandering, gradually transforming distribution of temporarily sustained representations?

References:

Amaral DG. 1987. Memory: Anatomical organization of candidate brain regions. In: Handbook of Physiology; Nervous System, Vol V: Higher Function of the Brain, Part 1, Edited by Plum F. Bethesda: Amer. Physiol Soc. 211-294.

Baars, Bernard J.(1997), In the Theater of Consciousness (New York, NY: Oxford University Press)

Baars, Bernard J. (2002) The conscious access hypothesis: Origins and recent evidence. Trends in Cognitive Sciences, 6 (1), 47-52.

Baddeley, A.D. (2000). "The episodic buffer: a new component of working memory?".Trends in Cognitive Science4: 417–423.

Baddeley, A.D. (2007). Working memory, thought and action. Oxford: Oxford University Press.

Carpenter, G.A. & Grossberg, S. (2003), Adaptive Resonance Theory, In Michael A. Arbib (Ed.), The Handbook of Brain Theory and Neural Networks, Second Edition (pp. 87-90). Cambridge, MA: MIT Press

Chalmers, D.J. 1995. The conscious mind: in search of a fundamental theory. Oxford University Press

Chalmers, D.J. 2010.The Character of Consciousness.Oxford University Press.

Cowan, N. (2005). Working memory capacity. New York, NY: Psychology PressCrick F, Koch C. A framework for consciousness.Nature Neuroscience. 6(2): 119-126.

Crick F & Koch C. 2003. A framework for consciousness. Nature Neuroscience. 6(2):119-126.

Damasio AR. Time-locked multiregional retroactivation: A systems level proposal for the neural substrates of recall and recognition. Cognition, 33: 25–62, 1989.

Fuji H, Ito H, Aihara K, Ichinose N, Tsukada M. (1998). Dynamical Cell Assembly Hypothesis – Theoretical possibility of spatio-temporal coding in the cortex.Neural Networks. 9(8):1303-1350.

Hebb, Donald (1949). The Organization of Behavior. New York: Wiley.

Horton JC, Adams DL. (2005) The cortical column: a structure without a function. Philos. Trans. R. Soc Lond. B Biol Sci 360 (1456): 837-862.

Jackendoff, R. 1996. How language helps us think. Pramat. Cogn. 4, 1-34.

Johnson-Laird PN. 1998. Computer and the Mind: An Introduction to Cognitive Science. Harvard University Press.

Klimesch W, Freunberger R, Sauseng P. Oscillatory mechanisms of process binding in memory. Neuroscience and Biobehavioral Reviews. 34(7): 1002-1014.

Lansner A. Associative memory models: From the cell-assembly theory to biophysically detailed cortex simulations. Trends in Neurosciences. 32(3):179-186.

McNaughton, B. L. (1991). Associative pattern completion in hippocampal circuits: New evidence and new questions. Brain Res Rev 16, 193-220.

Meyer K, Damasio A. Convergence and divergence in a neural architecture for recognition and memory.Trends in Neurosciences, vol. 32, no. 7, 376–382, 2009.

Miller G. 1956. The magical number seven, plus or minus two: Some limits on our capacity for processing information. The Psychological Review. 63, 81-97.

Miller EK, Cohen JD. 2001. An Integrative Theory of Prefrontal Cortex Function. Ann Rev Neurosci 24:167-202.

Moscovich M. Memory and Working-with-memory: A component process model based on modules and central systems. Journal of Cognitive Neuroscience. 4(3):257-267.

Moscovitch M, Chein JM, Talmi D & Cohn M. Learning and memory. In Cognition, brain, and consciousness:  Introduction to cognitive neuroscience. Edited by BJ Baars& NM Gage. London, UK: Academic Press; 2007, p.234.

Edelman, G. Neural Darwinism: The Theory of Neuronal Group Selection (Basic Books, New York 1987).

Edelman, G. Second Nature: Brain Science and Human Knowledge (Yale University Press 2006)

Edelman, F, Tononi G. (2000). A Universe of Consciousness: How Matter Becomes Imagination. Basic Books.

Fuster JM. 2009. Cortex and Memory: Emergence of a new paradigm. Journal of Cognitive Neuroscience. 21(11): 2047-2072.

Goldman-Rakic PS. 1995. Cellular Basis of Working Memory. Neuron. 14: 477-485.

Gurney, KN. Reverse engineering the vertebrate brain: Methodological principles for a biologically grounded programme of cognitive modeling. Cognitive Computation. 1(1) 29-41.

Pereira A Jr, Ricke H. 2009. What is Consciousness?: Towards a Preliminary Definition. Journal of Consciousness Studies. 15(5):28-45.

Reser, Jared. What Determines Belief: The Philosophy, Psychology and Neuroscience of Belief Formation and Change. Verlag Dr. Muller. Saarbrucken, Germany. 2011.

Reser, JE. Assessing the psychological correlates of belief strength: Contributing factors and role in behavior. Doctoral Dissertation, University of Southern California. Usctheses-m2627.

Reser, Jared. “The Neurological Process Responsible for Mental Continuity: Reciprocating Transformations between a Working Memory Updating Function and an Imagery Generation System.” Association for the Scientific Study of Consciousness Conference. San Diego CA, 12 July 2013.

Shear J. 1997. Explaining consciousness: The hard problem. Cambridge Massachusetts: MIT Press.

Tononi, G. 2004. An information integration theory of consciousness. BMC Neuroscience. 5:42.

Vimal R. 2009. Meanings attributed to the term consciousness: An overview. Journal of Consciousness Studies. 16(5):9-27.