Machine Learning & Data Mining

I'm predicting an output based on input data. These input data are measurements of a system that can be in three different modes, for instance some machine that has three pre-determined configurations. The mode is known for each set of input data.

If I want to make a machine learning algorithm, the received wisdom, at least as I've received it, is that you should not train one e.g artifical neural network (ANN) on all the data, but train one ANN on each of the three modes. This will help with training and generalisation since the examples the ANN encounter will be more similar.

This seems straightforward, but I have trouble showing it. For instance: AFAIK: The number of examples needed for a given performance of an ANN scales with the number of weights. so if the number of weights can be reduced to one third, you'd get the same performance using only a third of the training examples.

An ANN with 5 inputs, 12 hidden nodes and 1 output has 72 weights. I use 90 examples, with 30 examples of each mode, to train it to a given performance.
I then make three smaller ANNs. Each with 5 inputs, 4 hidden nodes and one output. Each net now have 24 weights (a third of 72) and so only require 30 examples to give the same performance. But since I'm only feeding examples with one mode to each ANN, I still need 90 examples to train all three correctly! And this is assuming that the modes didn't "share" hidden nodes in the orginal ANN, so that splitting up in 4*3 hidden nodes makes sense.

Should I ditch this received wisdom, or have I simply misunderstood when it should be applied?

I am curious if anybody did any work on classifiers that could be trained on a dataset where each sample belongs to more than one class and then predict not just a single class for each test sample, and even not just a set of options with different confidence, but associate a confidence with every (or just some) sub-set of classes in relation to each test sample.

For example if we have pictures as samples and three possible labels could be attached to each picture: beach, water, desert, it could say for some sample that it would give high confidences to (desert), (beach, water), (beach), lower confidence to just (water), and very low confidence to (desert, water) and (desert, beach)? The example is not very good, but I hope it explains what is it I am looking for.

I am just interested in some reading on the topic. Thank you.

When could I find simple real-life examples of the usage of the theory of statistical machine learning?

As I learned from Vapnik V.N., "The Nature of Statistical
Learnig Theory" (1999), there is a classical task:

There are two assymmetric dices. There is large list of results of
independent trials (pairs of data) for the dices. How to calculate the
propability distribution for each dice which altogether will give the
propability distribution of sum of points of dices as close as
possible to propability distribution function which we already know
from other source (so the results of trials and the propability distribution function of sum are known. The propability distribution function of points in dices are required)?

Just wanted to say thanks for making a community around one of my fave subjects :)

I've been trying to make a neural network learn a relatively simple problem, but no luck. So I thought I'd hear if anyone's got any ideas.
( explanation with pictures )

Machaon Cluster Validation Environment (Machaon CVE) is intended for application of different clustering and validation algorithms to experiment gene expression data. It directs to partition samples or genes into the groups characterised by similar expression patterns and evaluate the quality of the clusters obtained.

The software is implemented as a multi-window Java application, which allows working with different datasets, clustering and validation algorithms, and results simultaneously.

The developed software system may be effectively used for clustering and validating not only DNA microarray expression analysis applications, but also other biomedical and physical data with no limitations.

The information about the system and free downloads may be found at http://machaon.karanagai.com

Came across a nice "Tutorial on Support Vector Machines for Pattern Recognition" by Christopher J.C. Burges of Microsoft Research. It is published in 1998, but would be still quite useful to everybody who gets himself into the field of SVMs.

I am currently dealing with application of 1-class SVMs to image classification in incremental and also active learning setting. Could anybody suggest any papers to read on such SVMs, both purely theoretical and in application to any field, please?

(crossposted at ai_research)

http://hunch.net/ is a great machine learning blog.

Google Calendar is a great tool. It's a pity it does not work well with Outlook and Pocket Outlook.

Anyway, I've created a calendar of Machine Learning, Data Mining and other relevant conferences in GCal and going to populate it further over the time. It is public, so you can view the events. If you want to participate in population of the calendar, please do contact me, I will share it with your GCal account.

Let say we have a large enough image library. Consider the following image retrieval scenario:

Some user is looking for an image for his web site design, for example. The easiest UI for it is a text search, so the user types the keywords and retrieves some images as a result.

The problem here is that the association of keywords with images is usually quite poor in such libraries. The difference in vocabularies used by the contributors of images and the searcher is also an issue. Thus the user may not necessarily obtain the results that match his needs well enough.

At this stage, CBIR can be used. It is a nice approach, allowing the user to select what images from the result match his needs best (and, in some techniques, also what do not match it at all), and force the system to return a new result set matching the needs of user better than the original one.

So in few CBIR iterations the user finds what he needs.

Let me repeat it: first one step of a text search, then few steps of CBIR.

Now comes the obvious idea: what if we keep the keywords of the initial text search query as tags associated with the images marked by user as relevant? In that case we actually make not only the contributors, but also the majority of users of such a library to tag images in it, which enforce the text searching, possibly reducing the required number of CBIR steps.

I am not dealing with CBIR myself, so I did not come across any publication describing such an idea so far. Although I am pretty sure some must exist.

The other interesting resource is MLpedia, a Wiki-based Machine Learning encyclopedia project started by John Winn from Microsoft Research @ Cambridge. Few other individuals are contributing to it. The number of articles is not that large yet, with quite a few stubs instead of full articles, but it is open for contributions from anybody, so it may become a vaulable resource over the time.

This is the list of journals somehow relevant to my current field of research (image classification in folksonomies) taken from the recent post on my own blog. I ranked them here according to their relevance to my research:

1. Pattern Recognition


2. IEEE Transactions on Pattern Analysis and Machine Intelligence


3. Journal of Machine Learning Research


4. Machine Learning


5. Artificial Intelligence Review


6. Artificial Intelligence


7. Data Mining and Knowledge Discovery


8. Journal of AI Research


9. IEEE Transactions on Knowledge and Data Engineering

The relevance is estimated as a number of relevant papers found in the issues between January 2005 and now.

I thought the list may be useful for somebody else.

I've recently noticed that there is a substantial number of people in the Live Journal, who expressed some interest in the fields of machine learning and data mining. Doing some research in ML myself, I guess it would be useful to have some kind of community blog devoted to these two disciplines, that are closely related to each other.