Google Search Console

Google Search Console integration guide

Google Search Console is a free tool suite from Google that helps you measure your site's search traffic and performance.

In this guide we'll show you how to create a timeline in Clarify that visualizes your search analytics data.

Visualize your Google Search data in Clarify!Visualize your Google Search data in Clarify!

Visualize your Google Search data in Clarify!

👍

Setup your site in Google Search Console

You can easily verify your site and start using the tool for free.
Visit Google Search Console

After you have verified and added your site(s), you'll have access to your Performance Report which contains analytics about how your page perform in Google Search.

Brief description of the performance data we'll focus on in this guide:

Total clicks
How many times a user clicked through to your site.

Total impressions
How many times a user saw a link to your site in search results.

Average CTR
The percentage of impressions that resulted in a click.

Average position
The average position of your site in search results, based on its highest position whenever it appeared in a search.

Example Google Search Console performance reportExample Google Search Console performance report

Example Google Search Console performance report

Getting Started

At this point, we'll assume that you have verified your site in Google Search Console (GSC) and is up and running and ready to go.

1. Authentication

We use Google Search Console APIs to retrieve our data. Every request to the search GSC needs to be authenticated so you need to use OAuth 2.0 to achieve this.

OAuth 2.0 scopes for GSC API
read/write access: https://www.googleapis.com/auth/webmasters
read-only access: https://www.googleapis.com/auth/webmasters.readonly

If you use Google Cloud Platform to manage the email accounts for your company you might need to setup new credentials, make sure that your user has access to the Google service and that the APIs is enabled.

2. Node-RED setup

If you are new to Node-RED or haven't checked out our Node-RED guides yet, we recommend that you check out Introduction to Node-RED before continuing.

Install google oauth2 node-red node

In this guide we'll use the node-red-contrib-google-oauth2 (unofficial) node to authenticate and communicate with Google.

In your node-red dashboard select Manage palette from the main top right hamburger button. View the Install tab and search and install the node as displayed in the screenshot below.

Installing OAuth2 node in Node-REDInstalling OAuth2 node in Node-RED

Installing OAuth2 node in Node-RED

If you have successfully installed the node you'll now have the node available in the left sidebar in Node-RED as in the screenshot below.

Node-RED Google nodeNode-RED Google node

Node-RED Google node

Configuring the Google node

Drag the Google node to your workflow, double-click, and select the edit button for "Connection".
Enter your Client ID and Client Secret which you obtained as part of the authentication step.
In this example, we use the scope for read-only access.
When you are ready you can press "Start Authentication" to authenticate with your Google account.

Google node credentials setupGoogle node credentials setup

Google node credentials setup

In this example we'll use the webmasters:v3 API and the operation named searchanalytics.query.

3. Setup Query

In this step we'll build the query which is used by the API to request the data we need. You can find more information in the Google API Reference.

Node-RED flow with Inject, Build Query & Google nodesNode-RED flow with Inject, Build Query & Google nodes

Node-RED flow with Inject, Build Query & Google nodes

Start by dragging an Inject node and "Function"-node from the left sidebar to your workflow and connect them.

📘

Tip

In Node-RED you can drag a "Debug"-node and connect it to nodes to display the output data from that node. In the screenshot above you could connect the "Debug"-node to the "Build Query"-node to preview the output before connecting it to the "google"-node.

Double-click on the Function node and rename it (E.g. to Build Query) and edit the following example Javascript to fit your use-case. You need to at least edit YOURSITE with the domain name of your site in GSC.

let now = new Date();
now.setDate( now.getDate() + 1 );
let endDate = new Date(now).toISOString().split("T")[0];
now.setDate( now.getDate() - 7 );
let startDate = new Date(now).toISOString().split("T")[0];

msg.payload = {
    "startDate": startDate,
    "endDate": endDate,
    "siteUrl": "sc-domain:YOURSITE",
    "dimensions": [
        "date"
    ]
}

return msg;

In this example, we create two dates.
endDate: One day ahead of today
startDate: 6 days before today

We specify the siteURL so that the API knows which entity we want to retrieve data from. Further, we use the dimension date so that the data rows are returned by date.

The query is applied to the msg.payload object because this object is used by the Google node in the next step to perform the API call.

When you have tested that the query looks Ok, connect the Build Query node to the Google node.

4. Prepare Data

In this step, we'll verify the data response from the Google API and prepare the data for Clarify.
Connect a Debug node to the Google node (make sure the node is configured with "Output complete msg object").
After you have connected the Debug node you can press the trigger button in front of the Inject node to trigger the flow.

Node-RED debug node trigger buttonNode-RED debug node trigger button

Node-RED debug node trigger button

A short summary of our current progress before we look into the output from Google. We now have:

  1. The Inject node which triggers the flow
  2. Build Query node which creates the query parameters we send to the Google API.
  3. Google node to authenticate and perform Google API requests.
  4. Debug node to display the result from the Google API.

Below you'll find a sample response from Google which we will re-format and send to Clarify.

{
   "_msgid":"ad6a7798.d8ad88",
   "payload":{
      "rows":[
         {
            "keys":[
               "2021-04-22"
            ],
            "clicks":2,
            "impressions":188,
            "ctr":0.010638297872340425,
            "position":21.0531914893617
         },
         {
            "keys":[
               "2021-04-23"
            ],
            "clicks":1,
            "impressions":112,
            "ctr":0.008928571428571428,
            "position":25.919642857142858
         },
         {
            "keys":[
               "2021-04-24"
            ],
            "clicks":1,
            "impressions":55,
            "ctr":0.01818181818181818,
            "position":34.36363636363637
         },
         {
            "keys":[
               "2021-04-25"
            ],
            "clicks":0,
            "impressions":73,
            "ctr":0,
            "position":34.42465753424658
         }
      ],
      "responseAggregationType":"byProperty"
   }
}

To continue, drag a Function node to your workflow, double-click the node and name it "Prepare Data". Again, you can drag a Debug node and connect it to your newly created node to verify that the data is formatted correctly before being sent to Clarify.

Node-RED with newly added Prepare Data and Debug nodesNode-RED with newly added Prepare Data and Debug nodes

Node-RED with newly added Prepare Data and Debug nodes

Example Javascript which can be used in the Prepare Data node to format data for Clarify.

let records = msg.payload.rows;
let dataFields = ["clicks", "impressions", "ctr", "position"];

if (records.length === 0) {
    return null;
}

dataFields.forEach((field) => {
    let dates = [];
    let values = [];
    records.forEach((row) => {
        let dateString = new Date(row.keys[0]).toISOString();
        let value = row[field] ? row[field] : 0;
        dates.push(dateString);
        values.push(value);
    });
    
    msg.topic = "google-search-" + field;
    msg.signal = {
        name: "Google Search " + field,
        labels: {
            "data-source": ["Google Search Console"],
      }
    };
    msg.payload = {
      "times": dates,
      "value": values,
    }
    
    node.send(msg);
});

return null;

The code above will send a message for each of the data attributes we want to create a signal for in Clarify.

5. Send data to Clarify

In this step, we'll send the actual data to Clarify. As a preparation, you need to follow the guides Installing the Clarify nodes and Using the Clarify Insert node to make sure your workflow is able to correctly send data to Clarify.

When you have installed the Clarify nodes and setup credentials you can connect the Prepare Data node to the Clarify node. We also recommend that you connect two Debug nodes to the Clarify node. One for Errors and the other for other debug messages.

Final flow in Node-REDFinal flow in Node-RED

Final flow in Node-RED

📘

Automatically trigger flow

The Inject node supports automatic triggering at certain intervalls. You can achieve this by double-clicking on the Inject node and configuring the Repeat setting. For example by setting "repeat interval" with "1 hour" the workflow above will trigger and update data every hour.

6. Visualize data in Clarify

When our workflow has run we now have 4 new signals in Clarify which contain data for Clicks, Impressions, CTR & Position.
To visualize the data we need to publish the signals as Items and create a new timeline.

If you follow the guide, you can now create a new timeline with our fresh data. It might look similar to the screenshot below.

Fresh items available in Clarify Item BrowserFresh items available in Clarify Item Browser

Fresh items available in Clarify Item Browser

Drag the Google Search Console Items to the timeline and personalize the timeline with labels and colors that suit you and your company.

Timeline in Clarify with Google Search Console dataTimeline in Clarify with Google Search Console data

Timeline in Clarify with Google Search Console data

Congratulations

You have now made your Google Search data more available and easier to explore. The next steps in Clarify would be to invite your teammates to the timeline so that you can collaborate around this data.


Did this page help you?