This is unsupervised learning project in the SJSU data mining course. Students need to classify the sentences (already numerized by the SJSU course Prof) into clusters using Kmeans algorithm. Below is the performance of students in the class. Seems not so bad.   The whole project includes 3 parts: data preprocess, Bisec Kmeans and the basic Kmeans algorithm. 1. Data preprocess The clustering documents have 27673 terms in total. That is said number of features are 27673. Here is an excerpt of the sample data.  First, we will buildRead More

This is a SJSU course project.  We need to generate rule from Criminal data to see any criminal activity cause issue. The data It is got from the official website DataSf. The features It only has several useful features for the association rule generation. Those are Category, Descript, DayOfWeek, Data/Time, PDDistrict, Resolution and Address. Below is some sample data. The data Preprocess As some of the features in the above are similar, or not yet good higher extract, we should pre-process the data before we go into next algorithm step.Read More

This is a supervised learning project. The Text Sentiment Analysis is combined with three parts, the text data preprocess, the text data representation, three models (CNN, KNN,LSTM). This is an SJSU course project. I use the python to implement those model and generate the result. Here is the detail. the data preprocess Here is the sample data. The left side number is the target sentiment label, the right side text is the input text. From the text sample, more likely, those data is got from tweeter-similar samples by the professorRead More

The practice is to understand how Tensorflow applied to shadow NN in MNIST data. The practice is from Big Data University lectures. Reference: Support_Vector_Machines.html  (Coursera Machine Learning Course) Big Data University TensorFlow course Deep Learning Concept  Using multiple process layer with non-linear algorithm to simulate brain ability;  A branch of machine learning. We will focus on shadow NN in this note. Shadow NN MNIST Example: two or three layers only. In the context of supervised learning, digits recognition in our case, the learning consists of a target/feature which is toRead More

This Note is for image texture explanation: Reference  https://courses.cs.washington.edu/courses/cse576/book/ch7.pdf  (computer vision) Why Texture Texture gives us information about the spatial arrangement of the colors or intensities in an image. Why? The answer is the histogram can’t fully represent/classify images. All images below are half white and half black. However, the images are different. How to recognize texture Structural approach: Texture is a set of primitive texels in some regular or repeated relationship. Statistical approach: Texture is a quantitative measure of the arrangement of intensities in a region. Statistical method Co-occurrenceRead More

The note is to understand the concept/rise of CNN. Reference: http://colah.github.io/posts/2014-07-Conv-Nets-Modular/ Introduction convolutional neural network Lot’s of same neurons, similar as java function, which can be re-use X is the input layer (you can sense that is see/hear/smell, etc. for example, image, video, audio, document) Next Layer is not always fully connected with previous layer:  one neuron of type A neuron is not fully connected to each X.    B is not fully connected with All A  F is fully connected with all B Why so many same neurons? ThatRead More

The note is to describe how to calculate Convolution via manual or TensorFlow command. TensorFlow convolution common commands: y= np.convolve(x,h,”valid”) and y= np.convolve(h,x,”valid”)  are same…also true for “same”,”full” options. from scipy import signal as sg   sg.convolve is using FFT which is faster than np.convolve for big matrix convolution inverse = numpy.linalg.inv(x)  One dimension with zero padding When we apply kernel, we always use kernel’s inverse to multiply and sum. One dimension without zero padding  One dimension filter with multiple dimensions of input x= [[255,   7,  3],    Read More

The note intention is to understand the word2vec, and how to build a small NN to start your adventure on Deep Learning. You can see many source codes here to build the NN. But I am not yet built it with TF. Reference: A visual toy NN network for word2vec generation. The whole source code is from https://ronxin.github.io/wevi/.  The source code is written by javascript for NN with html and svg (https://www.tutorialspoint.com/d3js/d3js_introduction_to_svg.htm) for graph visibility.  Try your best to understand all thoroughly (not just well enough). It will help youRead More

To understand how TensorFlow works, we may need to learn a little bit the backend theory of some basic features. In my thought, the most wanted to know is the GradientDescentOptimizer feature. The core method of the feature is minimize. You can see the method everywhere in TensorFlow code. First, let’s check the official document first. Here is the link for for GradientDescentOptimizer. Minimize, https://www.tensorflow.org/api_docs/python/tf/train/GradientDescentOptimizer , copied here for detail. Add operations to minimize loss by updating var_list This method simply combines calls compute_gradients() and apply_gradients(). If you want to process the gradient before applying them, call compute_gradients() and apply_gradients() explicitlyRead More