Posted on

Journal Club: Week of 12/4/2015

A Few Useful Things to Know about Machine Learning
Pedro Domingos
Department of Computer science and Engineering University of Washington

This was an excellent read. I highly suggest this paper to novices and expert alike. Pedro goes through all the mysticism and what he calls “folk knowledge” in this paper. Knowledge that would takes years of machine learning you uncover. Pedro breaks down machine learning to simple concepts and shows the reader how to deal with them. Do not be mistaken, this is not a tutorial. You will not learn any new algorithms or application. You will only learn how to better use the ones you know. That being said, I believe it is best to go into this paper with a little background so you are not lost by what Pedro is explaining.
Pedro explores major pitfalls of people who are first learning machine learning as well as seasoned pros. I particularly liked his section on overfitting and the section on how to approach problems. ‘Start simple first” is a common piece of advice, but Pedro backs it up with examples and graphs showing how different methods perform. His advice on more data vs a clever-er model is invaluable. I highly suggest reading this paper, it is a quick and powerful read.

 

PLS-regression: a Basic Tool of Chemometrics
Svante Wold, Michael Sjostrom, Lennart Eriksson
Institute of Chemistry Umea University

 

Another paper on PLS, this one a little more current and a little more practical. Like Geladi’s paper on PLS, this paper goes in depth with PLS within the scope of chemistry and engineering, so its right up my alley.  After reading it, not all of my questions were answered butI felt like I had a better grasp on the algorithm. One thing I really liked about this paper was the diagnostics and the interpretation.
The paper is structured around an Amino Acid example. This serves as a good basis and testing ground as the provide the raw data for anyone to test on. The power of this paper is in the last couple of sections. The authors guide the reader through each step of interpreting the results. It goes through initial results to essential plots. Each plot gets its own subsection, however, they are not all given the same importance. The explanations on some of them are very brief, restricted to only one or two paragraphs.
If you are only going to read one section of this paper flip to the second to last page and read “Summary; How to develop and interpret a PLSR model.” Here the authors give a very quick overview which will get you on your feet and give you a basic understanding of what is going on. It makes as a good reference as well.
-Marcello
Posted on

Journal Club: week of 11/20/2015

A Few Useful Things to Know about Machine Learning
Pedro Domingos
Department of Computer science and Engineering University of Washington

This was an excellent read. I highly suggest this paper to novices and expert alike. Pedro goes through all the mysticism and what he calls “folk knowledge” in this paper. Knowledge that would takes years of machine learning you uncover. Pedro breaks down machine learning to simple concepts and shows the reader how to deal with them. Do not be mistaken, this is not a tutorial. You will not learn any new algorithms or application. You will only learn how to better use the ones you know. That being said, I believe it is best to go into this paper with a little background so you are not lost by what Pedro is explaining.

Pedro explores major pitfalls of people who are first learning machine learning as well as seasoned pros. I particularly liked his section on overfitting and the section on how to approach problems. ‘Start simple first” is a common piece of advice, but Pedro backs it up with examples and graphs showing how different methods perform. His advice on more data vs a clever-er model is invaluable. I highly suggest reading this paper, it is a quick and powerful read.

PLS-regression: a Basic Tool of Chemometrics
Svante Wold, Michael Sjostrom, Lennart Eriksson
Institute of Chemistry Umea University

Another paper on PLS, this one a little more current and a little more practical. Like Geladi’s paper on PLS, this paper goes in depth with PLS within the scope of chemistry and engineering, so its right up my alley. After reading it, not all of my questions were answered butI felt like I had a better grasp on the algorithm. One thing I really liked about this paper was the diagnostics and the interpretation.

The paper is structured around an Amino Acid example. This serves as a good basis and testing ground as the provide the raw data for anyone to test on. The power of this paper is in the last couple of sections. The authors guide the reader through each step of interpreting the results. It goes through initial results to essential plots. Each plot gets its own subsection, however, they are not all given the same importance. The explanations on some of them are very brief, restricted to only one or two paragraphs.

If you are only going to read one section of this paper flip to the second to last page and read “Summary; How to develop and interpret a PLSR model.” Here the authors give a very quick overview which will get you on your feet and give you a basic understanding of what is going on. It makes as a good reference as well.

-Marcello

Posted on

Journal Club: Week of 11/06/2015

Welcome to the first week of my journal club! I’ve gotten into the habit of looking for new and exciting papers to read. I made it a goal to read at least one new paper each week and I thought I’d share. The subject matter is not only on optimization but on various data analysis techniques, machine learning, multivariate controls, and other topics. Most of these papers are available online for free.


Multivariate Analysis
by Herve Abdi
University of Texas at Dallas


This paper serves as a what is best put as a catalog on multivariate techniques. It is by no means exhaustive, but contains a decent amount of techniques an brief blurbs on usage and statistical technique. The paper is organized by ones amount and format of ones data. First the author looks at techniques focused around one data set then he expands into two data sets. The two data sets section is split up into two categories. The first category assumes that one data set is trying to predict the other, the second category assumes that they are just different sets of variables. I like this organization as it makes it achieves the authors goal as a catalog. When I am looking for possible techniques, I can first look toward my data to rapidly see which techniques are not suitable.
When talking about the statistical techniques, the author goes into jsut enough details. The reviews rarely go above 2-3 paragraphs. One major criticism I have of this review is that it may be too brief. The author goes over how the techniques work, but barely touches upon possible issues or, more importantly, most appropriate usage. The author limits his discussion on usage to maximum 1 or 2 sentences.  However, this may be by design as the author mentions up front in the abstract that  “choice of the proper technique for a given problem is often difficult.” Nevertheless, this article starts as a jumping off point. If one desires to know more about the technique in question this paper at least gives a basis to expand upon.
Overall the paper is short, but provides enough insight for a reader to begin exploring possible options for multivariate analysis.


The paper can be found here.


Principal Component Analysis: Concept, Geometrical Interpretation, Mathematical Background, Algorithms, History, Practice
by K.H. Esbensen and  P. Geladi


This chapter by Esbense and Geladi fully guides the reader through the ins and outs of Principal Component Analysis (PCA). Because PCA is a basis and starting point for many multivariate methods, one needs a strong fundamental understanding. This chapter provides that and more. The chapter uses a geometrical interpretation of PCA which helps the reader to better understand what the algorithm does to decompose a series of variables and observations. Out of all the papers ive read on PCA, this chapter helped me the most.
this chapter includes an abundance diagrams which step the reader through all the projections PCA makes to our data set. Esbensen and Geladi take a matrix of variables and observations X, represented as a rectangle, and they run it through PCA algorithm.  This algorithm decomposes the matrix X into the two vectors, the loading , P, and scores , T, vectors. This is represented as the rectangle X decomposing into two smaller rectangles, T and P. They then go on to represent the “master equation” of PCA in the same way. This allows the reader to quickly grasp how PCA works visually. This is reinforced in the next section where PCA is represented as a change in coordinate axes. Finally, if geometric interpretations are not your thing, the authors include a simple algebraic approach, which stems into an algorithm for PCA. The algorithm is laid out briefly by the authors. I would have liked a more through step by step guidance, but this is satisfactory and enough to get a basic PCA program up and running.
Finally the chapter ends with an example and limitations. The example shows sample outputs and interpretation of the data which I found very beneficial. However, the example section is the weakest of the paper. I would have liked the authros to go into more detail of tha analysis and what conclusions you can make, these are briefly addressed (these may be contained in another section or chapter). Another thing I would have liked the actual data set to play around with, but i realize this was probably an expert chapter and shortened for space. Nevertheless, this paper is my go to for any questions or issues, but not on analysis of PCA.


Check out these two articles for an intro into multivariate!
Two new articles next week!
-Marcello