I’ve not been posting much lately, but I’ve been keeping busy. I’m working on several projects and an online course. I enrolled in a Machine Learning course on Coursera that focuses on Neural Networks. I’m finding the class material very interesting, but I frequently have to look up other material to make sense of things. As a complement to it I am reading about Linear Algebra concepts as they come up and watching Khan Academy videos on it. Regarding neural networks, I’ve found David Kriesel’s text a wonderful resource. You can find it on his website here. I only worked through Calculus 1 and 2 in school so it’s a different type of Math than to what I am accustomed. So far the hardest part to nail down is learning Linear Algebra notation and relearning partial derivatives.
I came across Neural Networks when I was an undergrad doing a research project on factors of Soil Genesis. Unfortunately I didn’t have the time to learn how to utilize them for the project. Maybe I’ll revisit that work after I’m done this course.
The first assignment involved using a Perceptron algorithm learn how to classify two types of points in R² (two dimensional space, think of an X-Y Coordinate plane).
The task was quite simple: the program takes different sets of the data and it must learn to group the types of data by drawing a line dividing the two types. The perceptron goes through iterations where it uses an initial guess and then revises its guess based on how much error there was between its guessed weights and the real weights. After a number of iterations your perceptron should be able to classify the two types of data if they are indeed linearly separable. In the image above you can see there are circles and squares and the perceptron’s first guess(black line) and the misclassifications in red.
And after a handful of iterations it correctly learns how to classify the data.
Of course there are ways to have two types of points where the data cannot be separated. You can see after a number of iterations the perceptron is unable to classify the points; The number of errors never converges to 0. It’s quite obvious by looking at the image, but you likely take your brain for granted.
This is pretty basic so far, but the concepts extend to truly interesting ideas. Neural networks can be used in Machine Learning for tasks in which a brain would excel. Things involving recognition—image classification, facial recognition, converting speech to text, recognizing objects in a picture, etc.. Sound interesting? I think so too!
In terms of programming I’m working on a web spider and search engine, both in Python, and both of which I hope to have in a final drafts before too long. For now I think I will be using Google App Engine for searching unless I run into any roadblocks. Eventually I’d like to take a look into high performance solutions—but I’ll save that until after the fundamentals.
I’m also working on a small game that will utilize some statistics concepts. I’m looking into maybe tying matplotlib into it to generate graphs that visualize chances(though that might make it too easy!). I’ve tentatively started this using flask, but I’m considering making it some sort of PHP-and-Python abomination.
Lastly, I’m tying together my first project using Leaflet. It puts stream data into a format where you can view it by clicking on streams on a map. I am using postgres for the database on this project, which my web host doesn’t provide, so for now it’s an internal facing project!
More another day!