Showing posts with label ML. Show all posts
Showing posts with label ML. Show all posts

Monday, 25 September 2017

How to use Hierarchical algorithm under Connectivity model of clustering ?



Methods of hierarchical linking algorithm
1. Min
2. Max
3. Ward's method
4. Group average

USE CASE
1. Text mining in NLP (Natural language processing) 
    For example : using Stanford NLP
2. Social network linking (LinkedIn,  FB)

EXAMPLE
from scipy.cluster import hierarchy
import matplotlib.pyplot as plt
import numpy as np

# Define your sample data set
ytdist = np.array([662., 877., 255., 412., 996., 295., 468., 268., 400., 754., 564., 138., 219., 869., 669.])

# Make Hierarchical linkage with data and  
# Single linkage (uses Min method)
# Possible values: single (uses Min) / complete (uses Max) / average
Z = hierarchy.linkage(ytdist, 'single')

# Plot and show the Dendogram showing the linkage
dn =
hierarchy.dendrogram(Z)

# plt.show()


How to create normalized histogram ?



Histogram : Graphical representation of the distribution of numerical data
It is used to understand how data is distributed across the bins.

Normal distribution : Arrangement of data set where most values cluster in the middle of the range and rest towards either extreme.
Its graphical representation is called BELL CURVE.
Mean, mode and median are all the same.

Normalized histogram : Histogram having Normal distribution.

EXAMPLE
import matplotlib.pyplot as plt
from numpy.random import normal, uniform


# Take some random datasets

# Create datasets with 1000 fractional numbers
gaussian_numbers = normal(size=1000)
# Create datasets with 1000 numbers in range of -3 to 3
uniform_numbers = uniform(low=-3, high=3, size=1000)

# Plot the histogram with dataset, no of bins = 20, type=step filled, in blue color
plt.hist(gaussian_numbers, bins=20, histtype='stepfilled', normed=True, color='b', label='Gaussian')

# Plot the histogram with dataset, no of bins = 20, type=step filled, in red color
plt.hist(uniform_numbers, bins=20, histtype='stepfilled', normed=True, color='r', alpha=0.5, label='Uniform')

# Set properties
plt.title("Gaussian/Uniform Histogram")
plt.xlabel("Value")
plt.ylabel("Probability")
plt.legend()
plt.show()

Sunday, 24 September 2017

How to create histogram ?


Histogram : Graphical representation of the distribution of numerical data
It is used to understand how data is distributed across the bins.

Normal distribution : Arrangement of data set where most values cluster in the middle of the range and rest towards either extreme.
Its graphical representation is called BELL CURVE.
Mean, mode and median are all the same.

Normalized histogram : Histogram having Normal distribution.

USE matplotlib TO PLOT THE HISTOGRAM (CHART)

EXAMPLE 1
import matplotlib.pyplot as plt 
# Plot the histogram with fixed data

plt.hist([1, 2, 1], bins=[0, 1, 2, 3])
plt.show()


EXAMPLE 2
import matplotlib.pyplot as plt
import numpy as np
 

# Generate items with 1000 items
x = np.random.normal(size = 1000)

# Plot the histogram
plt.hist(x, normed=True, bins=30)
plt.ylabel('Probability') 
plt.show()


EXAMPLE 3
import matplotlib.pyplot as plt
import numpy as np
 

# Generate items with 1000 items
x = np.random.normal(size = 1000)

# Cumulative histogram 1 
plt.hist(x,
         bins=100,
         normed=True,
         stacked=True,
         cumulative=True
)
plt.show()
 
# Cumulative histogram 2
plt.hist(x,
         bins=100,
         normed=True,
         stacked=True,
        
)
plt.show()