Exploring Google Trends Data with Python: A Step-by-Step Guide
Knowing what is trending is something that interests all of us. To understand how the topics we are interested in have trended throughout the year, we can use Python to gather and analyze the data. Like I certainly want to know how the Bharat Jodo Yatra trended throughout the year.
Getting Started with Pytrends
Our journey begins with setting up Pytrends, a Python library that interfaces with the Google Trends API. This is our primary tool for fetching the data. To kick things off, we initialize Pytrends and define the topics we're interested in.
You can replace 'Topic1' and 'Topic2' with any subjects you're curious about.
Fetching the Data
The core of our script is a loop that goes through each topic and fetches data on how much interest it has generated over time. This is done by sending a request to Google Trends for each topic.
The fetch_interest_over_time function is a Python script to obtain Google Trends data for a chosen topic. Key features include:
Parameters: It takes a
topic
, apytrend_obj
for Pytrends requests, aretry_count
with a default of 5 for the number of retries, and adelay
of 5 seconds before each retry.Process: The function executes a retry loop to attempt data fetching multiple times if errors occur. It configures the request with the topic, a specific timeframe, and a geo area (India).
Operation: It fetches the trend data and processes it by removing the 'isPartial' column.
Error Handling: The script prints error messages, pauses for the set delay, and retries fetching data.
Failure Handling: If all attempts fail, it signals failure and returns an empty DataFrame.
Loop through Topics: It iterates over each topic in a list named
topics
.Fetch Data: For each topic, it uses a function fetch_interest_over_time to retrieve Google Trends data for the specified timeframe and region.
Process and Combine Data: If data is successfully fetched, it renames the column for clarity and concatenates it with an existing DataFrame, creating a combined dataset that includes all topics.
Pause Between Requests: After processing data for each topic, it waits for 60 seconds before proceeding to the next one. This is likely to avoid hitting any rate limits of the API.
Then save the file in a CSV or Excel format which can be used to do data visualization using Tableau or Matplotlib library.
I imported it into Tableau and created plot for various topics in different sheets and then put them all together in a 1000 x 1500 dashboard.
You can interact with it on my Tableau Public Profile