[et_pb_section fb_built=”1″ custom_padding_last_edited=”on|tablet” _builder_version=”4.27.4″ _module_preset=”default” background_color=”#FFFFFF” custom_padding_tablet=”30px||30px||true|false” custom_padding_phone=”30px||30px||true|false” hover_enabled=”0″ global_colors_info=”{}” theme_builder_area=”post_content” sticky_enabled=”0″][et_pb_row _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_column type=”4_4″ _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n
This article was last updated on March 26, 2024 In the realm of web analytics, accurate data is paramount for informed decision-making. However, the presence of bots and irrelevant traffic can skew the insights derived from analytics platforms. To combat this, Google Analytics offers robust filtering mechanisms, including Bot Filtering and IP Filtering. With the recent transition to Google Analytics 4 (GA4), users might wonder how these filtering features differ from Universal Analytics (UA) and how they can leverage them effectively. In this article, we’ll delve into the nuances of bot and IP filtering in GA4, explore any disparities with UA, and provide insights on optimizing data integrity.<\/p>\n \u00a0<\/em><\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n In Early February 2021, we had a number of clients who saw unnatural spikes of traffic within their Google Analytics accounts. After a quick evaluation, we noticed that all of them had been hit with around 500 sessions from a \u201ctraffic-bot\u201d within the same week. As for a lot of people, it\u2019s not clear why this traffic is showing up and what\u2019s the purpose of this traffic we decided to write a quick blog post explaining what this is all about and how to make sure your Analytics accounts are set-up in a way that your metrics do not get flawed by these traffic bots.<\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/wearekemb.com\/wp-content\/uploads\/2023\/01\/What-is-bot-traffic-and-why-you-should-filter-it-out-bot-traffic-in-Google-Analytics.png” title_text=”What-is-bot-traffic-and-why-you-should-filter-it-out-bot-traffic-in-Google-Analytics” _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][\/et_pb_image][et_pb_text _builder_version=”4.19.4″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n Bot traffic is traffic generated by so-called \u201cbots\u201d which are automated programs that perform website activities such as website visits, clicks or the placement of comments. In the last months, there are a handful of providers who send Bot sessions as a means of doing Marketing for their services within your Google Analytics account. This is the latest Trend in analytics Spam (however, nothing new) and these services are offered as a way to influence your website metrics. As this \u201cunnatural and faked traffic\u201d can increase website visits, bounce rates and\/or time on site, some companies might think it\u2019s a good idea to make use of these services.<\/span><\/p>\n Bot Filtering serves as a shield against spam bots and irrelevant traffic, ensuring that analytics data accurately reflects genuine user interactions. In GA4, Bot Filtering is enabled by default, leveraging Google’s advanced algorithms to identify and exclude known bots and spiders from data collection. This automatic filtering helps maintain the integrity of analytics data without requiring manual intervention. We can however make a very clear statement that you should always refrain from using these services in order to \u201cmanipulate\u201d your website metrics.<\/strong> [\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_column type=”4_4″ _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_text _builder_version=”4.19.4″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.19.4″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]Next to bot traffic, there\u2019s another type of traffic that is accessing your page, however, should also not show up in your analytics reports. This traffic is generated by so-called spiders or web crawlers. You should not see any of this traffic showing up in your analytics data, however, you will be able to identify them in your server logs.<\/span><\/p>\n Crawlers access your website in order to access and process data. One of the most common spiders is the \u201cGoogle Spider\u201d which crawls the web in order to find \u201cnew content\u201d. However, other crawlers might be tools that you actively use for on-page optimization, website tracking or even something as simple as uptime monitoring.<\/span> Important!<\/strong> [\/et_pb_text][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n Every now and then we see common traffic spikes across the client accounts which are not covered within the standard bot and spider filtering. However, before you start excluding \u201dbot traffic\u201d you will need to identify which traffic should be excluded. Some of the most common indicators we have seen are:<\/p>\n Here is a recent example that was rather popular in the GA4 world as certain bot traffic started filling certain properties, bypassing GA4\u2019s ability to automatically filter bot traffic. By analysing the numbers it can be clearly deduced that this is not normal traffic, having thousands of users and sessions with almost no engagement at all.<\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/wearekemb.com\/wp-content\/uploads\/2024\/03\/bot-traffic-ga4-engaged-sessions.webp” title_text=”bot-traffic-ga4-engaged-sessions” _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][\/et_pb_image][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_column type=”4_4″ _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_text _builder_version=”4.19.4″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.19.4″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n Google is fully aware of the fact that there are many bots and spiders that visit your websites on a day-to-day basis and therefore had a standard feature built into Google Universal Analytics which was called \u201cBot Filtering\u201d. This feature can be found under the\u00a0<\/span>\u201cview settings\u201d<\/strong>\u00a0<\/span>within your “old” Universal Analytics admin panel. There you had the possibility to tick the field \u201cExclude all hits from known bots and spiders\u201d under the bot filtering section.<\/em><\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/wearekemb.com\/wp-content\/uploads\/2023\/01\/bot-filtering-google-analytics-1.png” title_text=”bot-filtering-google-analytics” _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][\/et_pb_image][et_pb_text _builder_version=”4.19.4″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n This automatically excluded all known box and spiders from your Google Analytics property starting from now.<\/span> [\/et_pb_text][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n The above described mechanism is one significant disparity between GA4 and UA in how bot traffic is handled nowadays. While in Universal Analytics, users needed to manually enable Bot Filtering through the admin settings, GA4 streamlines this process by automatically filtering out known bots without user intervention. This enhancement in GA4 simplifies the setup process and reduces the likelihood of inadvertently including bot traffic in reports.<\/p>\n IP Filtering allows users to exclude specific IP addresses or ranges from analytics tracking, effectively eliminating internal traffic, such as employees accessing the website, or unwanted traffic sources. While both GA4 and UA offer IP Filtering capabilities, the process of implementing filters varies slightly between the two versions.<\/p>\n While in UA, users navigated to the Admin panel, selected the desired view, and accessed the Filters section to create an IP filter, conversely, GA4 integrates IP Filtering directly into the reporting interface, offering a more intuitive approach. Users can define IP filters within the Analysis Hub or create custom segments based on IP addresses, granting greater flexibility in excluding unwanted traffic sources.<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n Bot and IP filtering play pivotal roles in maintaining the accuracy and reliability of analytics data in both Google Analytics 4 and Universal Analytics.<\/p>\n While GA4 streamlines the process with automatic bot filtering and integrated IP filtering, the fundamental objective remains consistent across both versions: to provide users with actionable insights based on authentic user interactions.<\/p>\n By understanding the nuances between GA4 and UA filtering mechanisms and implementing best practices, businesses can leverage analytics data effectively to drive informed decisions and optimize online performance.<\/p>\n [\/et_pb_text][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n One of the biggest changes between UA and GA4 is GA4\u2019s native ability to filter bot traffic automatically in comparison to manually setting this up in the previous version. However as we showed in our previous example, bot or unwanted traffic can still fill your GA4 property.\u00a0<\/p>\n What can be additionally done is to set up IP filtering in GA4. You can filter unwanted, internal or bot traffic if you are able to locate the IP in question.<\/p>\n First, head to your admin settings that you can find on the lower left hand side of your GA4 property, and go to Data Streams<\/strong>.<\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/wearekemb.com\/wp-content\/uploads\/2024\/03\/bot-traffic-ga4-admin-settings.webp” title_text=”bot-traffic-ga4-admin-settings” _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][\/et_pb_image][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n Second, head to Configure Tag settings:<\/strong><\/p>\n [\/et_pb_text][et_pb_image src=”https:\/\/wearekemb.com\/wp-content\/uploads\/2024\/03\/bot-traffic-ga4-tag-settings.webp” title_text=”bot-traffic-ga4-tag-settings” _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][\/et_pb_image][et_pb_text _builder_version=”4.24.2″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\n
<\/em><\/p>\nWhat is Bot traffic and why is it showing up in my Google Analytics?<\/h2>\n
<\/span><\/p>\n
These \u201cbot-sessions\u201d do not reflect actual user behaviour and will completely mess-up your actual website data. Again, we do not recommend anyone using these service as You want to optimize your website for actual users, and actual users only. Therefore it\u2019s important to keep your data clean and to make sure you exclude \u201cunnatural and unwanted traffic\u201d from your reporting.\u00a0<\/span><\/p>\nWhat is Spider traffic?<\/h2>\n
\nWithin the next steps, we will focus on bot traffic only, however, most of these rules can be applied for spider\/crawler traffic.<\/em><\/em><\/p>\n
\nBefore you continue \u2013 always make sure you have a \u201ctest-view\u201d within your Google Analytics account and test all of your changes in this test view before deploying anything in your production or reporting views.<\/strong>
\nNext to a testing view, we recommend that you have an \u201call website data view\u201d which contains all your raw data and does not contain any filtering.<\/em>[\/et_pb_text][\/et_pb_column][\/et_pb_row][et_pb_row _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_column type=”4_4″ _builder_version=”4.19.4″ _module_preset=”default” global_colors_info=”{}” theme_builder_area=”post_content”][et_pb_text _builder_version=”4.19.4″ _module_preset=”default” text_font_size_phone=”12px” global_colors_info=”{}” theme_builder_area=”post_content”]<\/p>\nHow to identify uncommon bot traffic<\/h2>\n
In order to exclude these, we suggest excluding them using a filter.<\/p>\n
There are few indicators for unnatural \/ bot traffic which you can search for within your Google Analytics property.<\/p>\n\n
How can you filter for known Spiders and Bots using Google Analytics?<\/h2>\n
Keep in mind that this filtering does not work retroactively.<\/em><\/em><\/p>\nChanges from UA to GA4\u00a0<\/strong><\/h3>\n
Bot Filtering in 2024<\/h2>\n
HOW IT\u2019S DONE IN GA4<\/strong><\/h3>\n