KCDCC Constructive Memory Group

I think Nick's presentation (for me at least) reinforced the idea that whatever it is we're each doing - analogy, design teams, optimisation, functional search (or not) - we're always going to be needing a situated conceptual system that is, essentially, the same.

I think a great place to kick off here is to outline what exactly we want from that underlying system that is generalisable to all of us. From there we can go straight to "how" and then then holy grail of "using what?" No illusions - the first implementations we do won't work to everyone's satisfaction, but without getting our hands dirty we won't know until it's too late.

I think we can all benefit from adopting Barsalou's method of talking about conceptualisation, and start defining our "core conceptual system" in terms of simulators and the re-enactment of perceptual symbols, but that's open for debate - if it doesn't fit someone's research, speak up now.

So, my first stab at the basics. Trying to stay as abstract as possible, we all need:

1. A way to learn about regularities present in sense data and the relationships and associations between them. This should be modality-specific but domain-general.
2. A way to construct "simulations" (John would say "experiences", I guess) that are interpretations of the agent's past experiences as they apply to current intentions in the current context (that is, other simulations that are "active").
3. An attentional system that both regulates construction (preventing it from becoming a "greedy algorithm" situation) and guides interpretation (selectively attending to interesting things).
4. A way to learn from the simulations the agent constructs; associations, actions (both internal and external and both successful and failed) particular interpretations and anything else should be accessible to the construction process in the future.

So, to me, that's our wish-list. Well enough there to occupy someone for five years, so I'm glad there're five of us with at least a year each.

So, anything else to add to the list? Big, small, anything? Disagreements with what I've got there or how I've said it?

Hi folks. I have an elementary Bayesian probability problem that is making me feel dumb. This comes from the first chapter of Judea Pearl's book on causality on pages 9-12. You have:

The prior odds of H from (1.16)
O(H) = P(H) / P(!H)

So I don't follow why the probability of H from (1.20) is
P(H) = O(H) / 1 + O(H)

I've juggled around algebra a whole lot and can't manage to form that equality. What is going on?

After our meeting on Friday I thought I'd post the DLM (Dynamic Link Matching) papers here.

Here's the original paper, or as original as I can find:

http://nn.cs.utexas.edu/web-pubs/htmlbook96/wiskott/

Here's the updated version that appears to be SOM-based, called FDLM:

ftp://ftp.neuroinformatik.ruhr-uni-bochum.de/pub/manuscripts/articles/ZhuMal2001a.ps.gz

And here's an interesting variant based on spiking neurons, called ODLM:

http://www-edu.gel.usherbrooke.ca/picr1601/oscillatory.pdf

Basically, they're all pattern-recognition-from-maps algorithms, with the useful property of being robust to affine transformations.

So, I don't know whether anyone's had any chance to look into the shapes problems or the more general ideas we were talking about two weeks ago, but I think we should meet again this week. Same time, 12pm Friday, same place.

If there's anything in particular people want to talk about then, maybe they could bring it up now? I want to talk a bit about particle systems, which I think are really interesting and we could maybe use them to make something resembling a Barsalou PSS if we can get over a few hurdles. I've attached a paper (Lee and Mumford, talking about particle systems and how they could be used to model the cortex) that I found interesting. You can skip over the maths in this if you feel inclined, I still got my head around it without reading the equations.

OK, no attach feature that I can see, here's a link: http://citeseer.ist.psu.edu/559717.html

Hi all, nice work on getting shapes done everyone.

I took last week off from doing work, so sorry for missing the meeting. Did I miss much? I have noticed that since the meeting posts on the board have dropped off, so I figure we should have another think about what it is about this that interests each of us.

Are people interested in using an open document that we work on to develop the paper that we think may result from this work? Hopefully you have all received an invitation in your inboxes so go and take a read and go edit-crazy on it. Make a start on the body text if you are feeling adventurous, even if it is just copy and pasting other work you have done so that we can all argue about it... (although quote and reference yourself even if it's just a research report so that we know we have to rewrite it...).

Enjoy!

Nick

This is a nice little collection of cognitive biases:

http://www.healthbolt.net/2007/02/14/26-reasons-what-you-think-is-right-is-wrong/

I mentioned at the meeting on Friday that a friend of mine has formed a Meetup.com group for people in Sydney interested in data mining, web mining, and natural language processing. They meet semi-regularly to talk about geeky stuff and generally to try and form an informal support network for people, mostly professionals but also academics, interested in the field. Next Monday there is going to be a meeting held at Google headquarters in Sydney, so if you guys are interested in meeting other people in the field, particularly in the private sector, and seeing what the mythical Google offices are like, I encourage you to come along. You can join the group and get the details of the event here.

Somwrita was asking me last Friday about some toy algorithms problems and I want to post my answers here. I've got a new answer to one of the problems. I'll use LJ cuts so if people are super keen they can try and work it out themselves before reading my answers.

1) Construct the most efficient algorithm you can that takes arbitrary strings like XXXABXAXXBB and finds the number of possible combinations of A followed by B, where there can be any number of intervening characters between the A and the B. For example, the previously mentioned string has 5 such combinations.

( Solution )

2) Construct the most efficient algorithm you can that takes a set of N numbers and generates all the products of the numbers in subsets with N-1 members. For example, { 1, 2, 3, 4 } would give 24, 12, 8, 6.

( Solution )

Somwrita: My solution to the second problem is different to what I gave you before and I think might be more efficient than the answer that you gave me if I remember yours right, plus also uses less memory since it only stores the last product, rather than several intermediate products.

If anyone has better solutions, commentary, or question please post them.

To summarise what we agreed on:

1. The input to our agents are closed shapes, expressed as a 200x200 bitmap.

2. Agents need to have some kind of generalised knowledge; see Barsalou (2005 Dynamic Interpretation paper) for a definition of summary representation that I think we should adopt for anything that we write up on this.

3. The agents respond to the queries:

       a. What does a shape look like
       b. What are some shapes like this one?

We will create five agents that all approach the problem in a different way, but that are all attempts to implement a constructive memory. Ideas used to represent concepts include:

    LSI, PSS, Shape grammars, Owen's one, and Kaz's one (I forget).

Within the next week we agreed put together at least ten random shapes saved as a 200x200 bitmap

Does anybody know of a site where you can get free online storage? (e.g. google, yahoo) and maybe store them there... We could also be optimistic and start a google open document to start writing the paper as we go (just a thought).

Greg has updated his idea for a Bayesian implementation of constructive memory. He says:

"I updated the notes from last week, in Section 1 of this weeks notes. Not finished though. I also added an example in Section 2. It also is unfinished. It turned out to be more difficult than expected."

I know how he feels.

This is just s sketch of an idea so all criticisms ideas and feedback are welcome.

One of the features of a perceptual symbol system (PSS; Barsalou 1999) is that it develops concepts on the basis of selective attention in interaction with the environment.

There are two levels of conceptual development:

1. The movement from sense data -> feature maps (i.e. a connectionist analog of the sense data internally) -> concepts that hold the 'features' or the selected important parts of these feature maps

2. The development of frames from holding links between concepts in space and time.

The web can be used to provide a nice example of the selective attention part of this (I think...)

Web pages have both 'physical' structure and code structure. A user interacts with web pages with an agent looking over there shoulder (the user helps us break away form having agents be motivated or anything).

Now, the whole web page onscreen is visible, as is the code (to the agent). But attention is only paid to those things that are being clicked.
So the only things that go towards the development of concepts and frames are those parts of the site that you paid attention to.

And then this is used for something... I am just not sure what. Am I making sense?

Thanks.

As discussed, lets get together face to face on Friday to talk things over. Rob has the staff meeting with KCDC on Friday at 1pm but he said he can come if we make it 12pm.

So if that suits everybody let's say meet somewhere between Rob's office and our labs and walk somewhere nice for lunch.

Nick

Heya all, I found a useful journal called "cognitive processing". It has a Barsalou paper in the to-be-released version that is available online at:

http://www.springerlink.com.ezproxy2.library.usyd.edu.au/content/110966/?sortorder=asc&Content+Status=Accepted

But the whole issue is dedicated to anticipatory learning, which some of us are interested in.

-Nick

Now that this has been switched over to a community I'll repost Nick's original message:

The way that I see this journal being used is as a discussion area for looking at ways that we can collaborate, and things that we can get up and running to support collaboration.

Ideas that we talked about were having:

- a central repository for code (can owen or kaz volunteer to look into rubyforge for feasibility? www.rubyforge.org)
- start on some kind of primitive development of a Barsalou-type model that satisfies our own curiousity

Does anybody have any other ideas?

I plan to hassle people about this if I don't hear back from them because unless we keep it alive we will all get distracted in our own worlds.

-Nick