NNC Neural Network Pioneer Award
Please join me to congratulate Prof. Leon Chua on his election as the year 2000 Neural Network Pioneer Award.

Mary Lou Padgett, Award Committee Chair
Gary G. Yen, Neural Network Technical Committee Chair

New Book Announcement
Title: Advances in Independent Component Analysis
Editor : Mark Girolami
Publisher : Springer Verlag ISBN 1-85233-263-8

Since 1995 the neural computing research community has produced a large number of publications dedicated to the Blind Separation of Sources (BSS) or Independent Component Analysis (ICA). This book is the outcome of a workshop which was held after the 1999 International Conference on Artificial Neural Networks (ICANN) in Edinburgh, Scotland.  Some of the most active and productive neural computing researchers gathered to present and share new ideas, models and experimental results and this volume documents the individual presentations.

Individual Chapter Titles
1. Hidden Markov Independent Component Analysis; Authors : W. Penny, R.Everson, S. Roberts
2. Particle Filters for Non-Stationary ICA; Authors : R. Everson, S.Roberts
3. Analysing the Independence of the Components by Topography; Authors : A.Hyvarinen, P. Hoyer, M. Inki.
4. Dependent Component Analysis; Author : A Barros
5. Ensemble Learning; Authors : H. Lappalainen, J. Misken
6. Bayesian Non-linear ICA by MLP's; Authors: H. Lappalainen, A. Honkela.
7. Ensemble Learning for Blind Separation and Deconvolution; Authors : J.Misken, D. MacKay.
8. MUCICA for Rank-Deficient Distributions; Authors : F. Palmieri, A.Budillon.
9. Blind Separation of Noisy image Mixtures; Author : L.K. Hansen.
10. Searching for independence in Electromagnetic Brain Waves; Authors : R.Vigario, J. Sarela, E. Oja.
11. ICA on Noisy Data : A Factor Analysis Approach; Author : S Ikeda.
12. Analysis of Optical Imaging Data Using Weak Models and ICA; Authors :J. Porril, J. Stone, J. Berwick, J. Mayhew, P. Coffey.
13. Independent Components in Text; Authors : T. Kolenda, L. Hansen, S.Sigurdsson.
14. Seeking Independence Using Biologically-Inspired ANN's; Authors : P.Lai, D. Charles, C. Fyfe.

PDP++ Software, Version 2.0
The PDP++ software is a neural-network simulation system written in C++.  It represents the next generation of the PDP software released with the McClelland and Rumelhart "Explorations in Parallel Distributed Processing Handbook", MIT Press, 1987.  It is easy enough for novice users, but very powerful and flexible for research use. The current version is 2.0, released August, 2000, which is a major upgrade from previous versions, as detailed below.

The software can be obtained by anonymous ftp from: Anonymous FTP Site:    ftp://grey.colorado.edu/pub/oreilly/pdp++ *or* ftp://cnbc.cmu.edu/pub/pdp++/        *or*   unix.hensa.ac.uk/mirrors/pdp++/
For more information, see our web page:  WWW Page:   http://www.cnbc.cmu.edu/PDP++/PDP++.html

There is a 250 page (printed) manual and an HTML version available on-line at the above address.

The new features in 2.0 include:
---------------------------------
  o MS Windows platform fully supported (using CYGWIN environment)
  o Project View window for GUI onto project/processes & specs
  o Enviro View rewritten, GUI for event/pattern layout, etc.
  o Grid View rewritten, interactively configurable grid layout
  o Easy viewing of entire network weights in grid log
  o Easy cluster plot interface, displayed in graph log
  o GUI for interactive construction improved
  o Context-senstive help via "Help" menu on all objects (via HTML)
  o Lots and lots of bug fixes, minor improvements: every known way to crash software has been fixed!

Software Features:
==================
  o Full Graphical User Interface (GUI) based on the InterViews toolkit.  Allows user-selected "look and feel".
  o Network Viewer shows network architecture and processing in real-time, allows network to be constructed with simple point-and-click actions.
  o Training and testing data can be graphed on-line and network state can be displayed over time numerically or using a wide range of color or size-based graphical representations.
  o Environment Viewer shows training patterns using color or size-based graphical representations; interactive configuration.
  o Flexible object-oriented design allows mix-and-match simulation construction and easy extension by deriving new object types from existing ones.
  o Built-in 'CSS' scripting language uses C++ syntax, allows full access to simulation object data and functions.  Transition between script code and compiled code is simplified since both are C++. Script has command-line completion, source-level debugger, and provides standard C/C++ library functions and objects.
  o Scripts can control processing, generate training and testing patterns, automate routine tasks, etc.
  o Scripts can be generated from GUI actions, and the user can create GUI interfaces from script objects to extend and customize the simulation environment.

Supported Algorithms:
=====================
  o Feedforward and recurrent error backpropagation.  Recurrent BP includes continuous, real-time models, and Almeida-Pineda.
  o Constraint satisfaction algorithms and associated learning algorithms including Boltzmann Machine, Hopfield models, mean-field networks (DBM), Interactive Activation and Competition (IAC), and continuous stochastic networks.
  o Self-organizing learning including Competitive Learning, Soft  Competitive Learning, simple Hebbian, and Self-organizing Maps ("Kohonen Nets").
  o Leabra algorithm that combines error-driven and Hebbian learning with k-Winners-Take-All inhibitory competition.  Over 40 research-grade simulations available for this algorithm in association with new book: "Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain", O'Reilly & Munakata, 2000, MIT Press.

SVMTorch
A Support Vector Machine for Large-Scale Regression and Classification Problems
by Ronan Collobert (collober@idiap.ch) IDIAP, CP 592, rue du Simplon 4, 1920 Martigny, Switzerland

Description

SVMTorch is a new implementation of Vapnik's Support Vector Machine that works both for classification and regression problems, and that has been specifically tailored for large-scale problems (such as more than 20000 examples, even for input dimensions higher than 100).

Source Code

The source code is free for academic use. It must not be modified or distributed without prior permission the author. When using SVMTorch in your scientific work, please cite the following article:
Ronan Collobert and Samy Bengio, Support Vector Machines for Large-Scale Regression Problems, IDIAP-RR-00-17, 2000. (available at
ftp://ftp.idiap.ch/pub/reports/2000/rr00-17.ps.gz).

The software has been successfully compiled on Sun/SOLARIS, Intel/LINUX and Alpha/OSF operating systems. Your can download it from
ftp://ftp.idiap.ch/pub/learning/SVMTorch.tgz.

First, you should download  the source code from
ftp://ftp.idiap.ch/pub/learning/SVMTorch.tgz and the examples from
ftp://ftp.idiap.ch/pub/learning/TrainData.tgz. Put this two archive files in
the same directory, and decompress them with

zcat SVMTorch.tgz | tar xf -
zcat TrainData.tgz | tar xf -

It creates two new directories : "SVMTorch" and "TrainData".

Now, go in the "SVMTorch" directory and edit the Makefile. You should only
have to change the following lines, depending on your specific platform :

# C-compiler
#CC=gcc
CC=cc
# C-Compiler flags
#CFLAGS=-Wall -W -O9 -funroll-all-loops -finline -fomit-frame-pointer
-ffast-math
CFLAGS=-native -fast -xO5
# linker
#LD=gcc
LD=cc
# linker flags
#LFLAGS=-Wall -W -O9 -funroll-all-loops -finline -fomit-frame-pointer
-ffast-math
LFLAGS=-native -fast -xO5
# libraries
LIBS=-lm

The default configuration is set for a machine running with the Sun Workshop compiler. An alternate (commented) configuration is proposed for the GNU gcc
compiler.

Type  "make all" and pray.

It should compile without any warning.

For some platform, you could have to change the include files needed for "times", a non-standard function used by svm_torch. You would have to edit the file "general.h" and change the lines

#ifdef I_WANT_TIME
#include <sys/times.h>
/*#include <limits.h>*/
#include <time.h>
#endif

If it doesn't work or if you don't want to measure the time of the learning machine, just comment the line :
#define I_WANT_TIME

Note that in "general.h" you can comment the line
#define USEDOUBLE
in order to do the computations in float. IT'S A BAD IDEA : svm_torch needs precision.

If everything went well, you should have two programs : "svm_torch" and "svm_test". The first one is the learning machine and the second one is the testing machine. If you want to show all the options, just run svm_torch or svm_test without any parameter.

To test the program in classification, try :
svm_torch -v -ae ../TrainData/classif_train.dat ../TrainData/model_dummy

It takes less than two minutes on a 300Mhz computer. You should have around 914 support vectors (this number could slightly change depending on the precision of your machine).

To test the SVM on the train data, try :
svm_test -ae ../TrainData/model_dummy ../TrainData/classif_train.dat
You should have around 0.78% missclassified.

To test the program in regression, try :
svm_torch -v -ae -rm -st 900 -eps 20 ../TrainData/regress_train.dat
./TrainData/model_dummy
You should have around 597 support vectors.

Test the model with :
svm_test -ae ../TrainData/model_dummy ../TrainData/regress_train.dat
The mean squared error should be around 187.2.

Options

The general syntax of svm_torch and svm_test is
svm_torch [options] example_file model_file
svm_test [options] model_file test_file

Where "example_file" is your training set file, "test_file" is your testing set file and "model_file" is the SVM-model created by svm_torch.

All options are described when you launch svm_torch or svm_test without any argument. By default, svm_torch is a classification machine. If you want the
regression machine, use option -rm. You should always use option -v with svm_torch : it gives a current error during learning. This error is only an indicator. It can oscillate.

File format

There are two main input formats for "input_file" and "test_file" in SVMTorch : an ASCII format, and a binary one.

The ASCII format is the following:
<Number n of training/testing samples>  <Dimension d of each sample+1>
<a11> <a12> <a13> .... <a1d> <a1_out>
 .
 .
 .
<an1> <an2> <an3> .... <and> <an_out>

where <aij> is an ASCII floating point number corresponding to the j-th value of the i-th example and <ai_out> is the i-th desired output (in classification, it should be +1/-1).

With the same notation, the binary format is:
<Number n of training/testing samples> <Dimension d of each sample>
<a11>...<a1d> ....... <an1>...<and> <a1_out>... <an_out>
(First save the input table, then the output table, all in binary)

There is another special input format for svm_test, when you don't have the desired output. (To use with the -no option).
The ASCII version of this format is :
<Number n of training/testing samples>  <Dimension d of each sample>
<a11> <a12> <a13> .... <a1d>
 .
 .
 .
<an1> <an2> <an3> .... <and>

And the binary version is :
<Number n of training/testing samples> <Dimension d of each sample>
<a11>...<a1d> ....... <an1>...<and>