The train-test split technique is a way of evaluating the performance of machine learning models. Whenever you build machine learning models, you will be training the model on a specific dataset (X and y). Once trained, you want to ensure the trained model is capable of performing well on the unseen test data as well. …
The size of the data you can load and train in your local computer is typically limited by the memory in your computer. Let’s explore various options of how to deal with big data in python for your ML projects. In certain situations, you might have a hard time working on large data, that is …
How to deal with Big Data in Python for ML Projects (100+ GB)? Read More »
Granger Causality test is a statistical test that is used to determine if a given time series and it’s lags is helpful in explaining the value of another series. You can implement this in Python using the statsmodels package. That is, the Granger Causality can be used to check if a given series is a …
Decorators in python allow you to dynamically change the functionality of another function, without altering it’s code. What? Is that possible? Yes. This covers: 1. What is a decorator and how to create one? 2. Easier way to decorate functions 3. Class decorators 4. Problem with docstrings on decorated functions and how to solve. What …
Decorators in Python – How to enhance functions without changing the code? Read More »
Generators in python provide an efficient way of generating numbers or objects as and when needed, without having to store all the values in memory beforehand. Introduction You can think of Generators as a simple way of creating iterators without having to create a class with __iter__() and __next__() methods. So how to create a …
Generators in Python – How to lazily return values only when needed and save memory? Read More »
Iterable in Python is any object that can be looped over. In order for an iterable to be looped over, an iterable needs to be converted to an iterator using the __iter__() method. Once created, it is actually the iterator that gets iterated over. This is also what happens internally when you run a for-loop. …
Iterators in Python – What are Iterators and Iterables? Read More »
Any Python file with a .py extension is a Module in Python. A python package is a collection of such modules along with a __init__.py file. Let’s understand how to work with modules and packages with clear examples. Introduction When you work on python projects, it’s not a good practice to have all you python …
Python Module – What are modules and packages in python? Read More »
Object oriented programming is an effective way of writing code. You create classes which are python objects, that represented meaningful entities which defines its own behaviour (via methods) and attributes. Let’s understand what a class is and the concepts behind Object Oriented Programming in Python Everything you have encountered so far in Python, such as …
How to use numpy.random.rand() function ? numpy.random.rand() function is used to generate random float values from an uniform distribution over [0,1). These values can be extracted as a single value or in arrays of any dimension. In this article, you will learn about various use cases of this function. Structural overview of numpy.random.rand() Syntax: numpy.random.rand(d0, …
What is Partial Correlation and it’s purpose Partial correlation is used to find the correlation between two variables (typically a dependent and an independent variable) with the effect of other influencing variables being controlled. For example, if there are three variables ‘A’, ‘B’, ‘Z’, If you want to find the relationship between ‘A’ and ‘B’ …
What is chi-square test and its purpose? Chi-square test was invented in the year ‘1900’ by the revered mathematician ‘Karl Pearson’. Chi-square test, also written as χ2 test is used to determine whether there is a statistically significant difference between the observed frequency and the expected frequency in one or more categories of the contingency …
Chi-Square test – How to test statistical significance for categorical data? Read More »
How to reshape a numpy array? The numpy.reshape() function is used to reshape a numpy array without changing the data in the array. It is a very common practice to reshape arrays to make them compatible for further calculations.
Use the pandas.DataFrame.sample() method from pandas library to randomly select rows from a DataFrame Randomly selecting rows can be useful for inspecting the values of a DataFrame.
Pandas DataFrame is used very popularly to store tabular data. Very often, we need to iterate over rows of the dataframe to perform various operations. This is a way of navigating the DatraFrame.
How to use Pandas Describe function? The pandas.describe function is used to get a descriptive statistics summary of a given dataframe. This includes mean, count, std deviation, percentiles, and min-max values of all the features. On applying pandas describe function to a dataframe, the result is also returned as a dataframe . This dataframe will …
The pandas.DataFrame.duplicated() method is used to find duplicate rows in a DataFrame. It returns a boolean series which identifies whether a row is duplicate or unique. In this article, you will learn how to use this method to identify the duplicate rows in a DataFrame. You will also get to know a few practical tips …
Pandas Series is a 1-dimensional array like object which can hold data of any type. You can create a pandas series from a dictionary by passing the dictionary to the command: pandas.Series(). In this article, you will learn about the different methods of configuring the pandas.Series() command to make a pandas series from a dictionary …
pandas.head() function is used to access the first n rows of a dataframe or series. It returns a smaller version of the caller object with the first few entries.
RegEx (Regular Expression) is a special sequence of characters used to form a search pattern using a specialized syntax While working on data manipulation, especially textual data, you need to manipulate specific string patterns. These may include retrieving hashtags from a tweet, extracting dates from a text, or removing website links. Pandas replace() function is …
