What is mobile-first indexing?
Mobile-first indexing means that Google has now started to index web content from a mobile-first perspective. In simpler terms, it will now be crawling, indexing and ranking the mobile version of your website pages, instead of the historically used desktop ones.
This update was originally introduced back in 2019 – if your website was submitted to Google crawler post July 1, 2019, it would have been indexed mobile-first by default.
For all other previously existing websites, Google’s been transitioning them over to mobile-first indexing over the last few years. This is slated to be finished by next month (July 2022), and is exactly the reason why understanding this algorithm change and implementing required corrections on your website is of absolute necessity right now.
To clarify, mobile-first indexing is happening for ALL web content. Google has not given us an option to opt in or out – and there is a very good reason why which we will cover in the next few sections of this article.
How is search changing with mobile-first indexing?
The way a search is run by Google has seen a lot of upgrades over the last decade. Historically, it was done by crawling and indexing text across websites, and keyword matching the ones that included keywords same as the entered search query. These results were then sorted in SERP based on keyword frequency in their text and relevance of inbound/ outbound links. Most digital folks would remember how keyword stuffing formed an important part of online content writing back then.
Google started learning synonyms to move away from the exploitation caused by keyword stuffing, and started awarding sites which showed a bit more naturally varied content. This also gave a wider understanding to Google of the meaning behind this content. However, the thing to note here is that it was still only matching the search query to keywords found in website texts.
When people found ways around this approach as well – now stuffing keywords and their synonyms in content pieces that flowed more naturally – Google had to move their focus on external off-page signals. This included PageRank signals, reviews and citations, backlinks and so on. A major benefit of this approach turned out to be the fact that they could now map relationships amongst different web pages and sites. Link and content relevance, domain age, authority, expertise, trustworthiness – everything started to play a much bigger role in how Google was ranking your website for a particular search query.
This led to the logical next step for Google of trying to move away from keyword or query matching to actually understand what is being searched. It paved the way for context development and understanding, kicking off with phrase-based indexing and further expanding with ontology-based query matching.
On a side note, if you’re interested in the technical side of how these work, I highly recommend fully reading the linked articles and other follow-up ones by Bill Slawski.
The next in line became the ground-breaking concept of entity understanding through device, language and location-based contexts (Cindy Krum talks about this phenomenon in detail in her article on entity-first indexing). Creating entities and understanding their connections with each other can allow Google to potentially figure out the search query context in terms of specific ideas and sentiments.
This is also where mobile-first indexing now comes into picture. By moving to a mobile-first index, Google is moving away from the older Link Graph to the brand-new Knowledge Graph. It is able to better understand how changing devices, languages, locations and time changes user preferences on what they are expecting a search query to return.
It is primarily driven through the creation and grouping of actual content into imagined entities, and establishing variable relationships among entities that change based on above factors. This raises the obvious question of how entities are defined.
What are entities?
Google defines an entity as “a thing or concept that is singular, well-defined, and distinguishable.” This means that it is a specific idea or concept that is universal in nature, and can have different forms based on certain defining variables.
One can use words, images, feelings, sounds and even abstract concepts to describe an entity. This leads us to believe that entities are language-agnostic by definition. Now this is a very important point when it comes to search because Google can use entities to sift through information without language constraints. It will no longer need to have a query translated to English, pick out the keywords, and fetch results based on this understanding. It can directly understand what’s being queried based on the user’s language, location, device, time, past behaviour (all of which can be considered as variables that make up an entity’s identity) – and fetch relevant results accordingly.
Let’s take a couple of quick examples to understand this better. If you search ‘lord of the rings movies’, Google Autocomplete gives you slightly different results depending on device. On a laptop, it shows more information-based suggestions, along with ‘people also ask’ and cast details (again all informational). It also provides for a possibility that the suggestions are not what you were looking for – in which case you can report them.
On a mobile phone, the suggestions are shifted to a more transactional phase. I’m assuming this is because Google feels we might be searching for transactional information such as where to watch LotR movies, what order to watch them in, what is the duration, etc. It does not provide any other information until you click through a particular query.
Now here’s something interesting. When we tried searching for ‘moon knight’ – which possibly should be similarly categorised – the search results are quite different. On the laptop, it provides a ton of informational content, but also gives you multiple results on where to watch. It actually is as specific as showing direct watch links for individual episodes.
However, while where to watch surfaces in the first-fold on mobile search, there is no further emphasis on it in the following sections. Surprisingly, there is a video result but it is just the trailer. If you wish to watch the episodes, you need to click on the orange ‘Episodes’ button in the horizontal navigation – increasing the clicks required to get to these.
The first search is about an entire movie series, not a particular movie – and so, the searches surrounding these are expected to be tilted towards information-based. On the other hand, the second search is specific to an episodic series – where each episode might not be a standalone entity – consequently, the search results are of a more navigational or transactional outlook.
These above examples showcase how minutely Google is defining these entities and their relationships with each other. It is able to correctly distinguish and index these based on the query and device context.
Entities have introduced a whole new way of looking at search using the Knowledge Graph Search API. Google is still definitely looking at inbound/ outbound links to understand relationships, but the concept of Link Graph is being faded out, with the importance associated with domains getting replaced by entities.
A great example of this is the sort of top of the page SERPs we see these days, which heavily comprise information that is not necessarily linked to any domain. Google shows local packs, people also ask, topic carousels, information cards and other such similar results that are purely information. The reason Google is able to surface these now is because of its capability to understand context beyond language.
What do understanding, relatedness and context mean for Google when it comes to entities?
Entities are understood primarily in terms of their characteristics. Now these characteristics can be a function of quality, time or space, and these functions can be matched with other characteristics (or entities) to form relatedness. Google’s patents often refer to these as nodes (entities) and edges (relationships).
Using this network of information, Google can break down a query to understand the context and subsequently, associate it with an entity. Queries can be mapped to one or more entities based on the entity association database (yes, Google does have an entity database!). Google uses context to understand which entity might be more relevant to the user based on the time, query framing, device, location, past behaviour and so on.
If we go one step further to understand this, there seems to be a very neat top-down hierarchy involved in creating and storing context. A vocabulary list gets created with each macro-context vector for every unique term from a domain. These lists are then subsequently used to classify newer pieces of information with the end objective of being able to form micro-context vectors.
It is surely simpler to unify this entity-based index using language-agnostic entities that have micro-context vectors associated for clarity than to comb through multiple variants based on user language, especially since translations are often not able to capture the true meaning of the original query.
The most noticeable difference here from Link Graph based results is the context-based changes in SERPs for the same search queries. An extremely common example that all of us would have seen – a ‘near me’ query done from your desktop might show regular text results along with local pack. However, the same query done through a mobile phone when you are on the road is bound to give you a higher number of local pack results – almost all of which will try to lead you to open Google Maps.
Another major win here is the ability to bypass language translations to understand queries. Entity-based indexing forms associations that are perhaps being matched to queries using a query score (this Google patent does refer to it, but I’m not sure if that is a definite thing they do). Even if a query score is not being generated, Google definitely has some relevance measure which they use to identify which entity is matching the query best, based upon the context information. It is also considering content freshness, click-through rate and follow-up queries based on the results we are seeing.
None of these steps, however, make any mention of translating the query to a base language and identifying possible results thereafter. This not only relates to that Google’s claim that BERT ‘will be able to return relevant results in the many languages Search is offered in’, but also their T5 text-to-text framework and the most-recent MUM update. For users, it basically means that the search results are going to be delivered faster and with higher context-based accuracy.
Is Google organising their index based on entities?
Absolutely. It is not a separate index, it’s the same index that’s been in use that is being re-organised. We have had multiple folks from Google confirm this in the last couple years.
It does not really make a lot of difference in our understanding of how entity-query associations are happening. The same index can have additional information regarding context and relationships associated with items, converting them to entities.
A particular item, which was defined using links before, can have multiple instances where context varies, creating context-based variants (which can be understood as brand-new entities themselves, instead of just a variant).
This might also be the primary reason why the roll-out for mobile-first indexing has been such a slow process. If Google is re-organising and categorising all the web content into entities, it is a huge amount of work that is being undertaken. There’s no way it could have completed it in a few months like its core updates.
Is your website mobile-optimised or just mobile-friendly?
Mobile-friendly and mobile-optimised are NOT the same. Your website could be showing up on mobile, but it might not necessarily be truly responsive. The content might be too long to read on phone, fonts just a tad bit smaller than required, images not resizing properly – there could be a ton of minor things which might affect user experience on mobile.
When your website is optimised for mobile, a lot of details are taken into consideration. Its loading speed (on slower networks as well), content or data heaviness, scripts minification, even the format and resolution in which images are served changes based on user agent and viewport of a particular device.
As mobile-first indexing being mandatory for all websites and scheduled for completion by July 2022, ensuring your website is fully optimised from a mobile-first approach is critical. It is not only a requirement to be able to rank on SERPs from here on, it is also a necessity in terms of how online interactions have progressed over the last few years.
Also, do note that Google WILL crawl and index your site pages irrespective of their responsiveness. As long as their smartphone Googlebot is able to access the pages, crawling is going to happen.
There should be no need for us to explain why having a fully responsive site is the only way you are going to rank prominently any longer. The possibility of your website seeing a dip due to it not being optimised from a mobile-first perspective is very much on the cards in coming months.
How do you improve your site for mobile-first indexing?
First and foremost, optimise your website for your users, and not for Googlebot. The latter is a controlled-environment run by Google to see how users might experience your website – so the end goal really is to provide a seamless experience.
There are some technical enhancements that you can work in to make it easier for Googlebot to render, crawl and index web pages, but your primary focus should definitely be on the content side of things.
1. Ensure your content is easy to read, relevant and ready for mobile-viewing
Take mobile screen sizes into consideration when you are developing content or deciding on its placement on your site. Big blocks of content are difficult to navigate on a small screen, and can break the thought flow for a user.
It would probably make way more sense to structure content across multiple pages if you have a lot of content to share around a topic. It can be divided into relevant sub-topics, with clear headings and presented to users in digestible bites.
Also, at the risk of iterating a super basic thing – do not have different content versions showing up on mobile and desktop. Now that Google is going to crawl mobile pages, what’s the point of having a separate desktop site anyway?
2. Content also extends to images, audio or video files on your website
It is always a good idea to compress media files and upload them in universally accepted formats so that there is no browser or device dependence.
Do consider the possibility that certain media files might not play as required on smaller devices, so your overall content experience should not be dependent on them. Providing alternate texts and/ or captions in unavoidable cases to continue the story flow is a sensible approach.
Have your web pages and media resources available for Google crawler, and see that no essential content is lazy-loading post user interaction.
3. Optimize your site for structured data and core web vitals, as well as look to minimise technical concerns
Structured data or schema markup has become a default pre-requisite to having your own website these days. Structured data helps Google understand various content categories and items on your web pages, and create entity-driven hierarchies based on these definitions. Consider what we have discussed how these hierarchies affect overall searchability of your content, and you will see the absolute importance of this step.
A sure-shot way to optimise is to run your website through Google’s Pagespeed Insights or (Lighthouse if you are a developer yourself) to understand how Googlebot is rendering it for mobile and desktop. This page experience report has been transformed to present data in form of Core Web Vitals – which roughly translates to how fast, stable and interactive your site is for users.
It also provides a ton of technical fixes you can implement to enhance user experience.
Quick note on core web vitals for a website with AMP pages – Google bases page experience signals on the AMP pages, but indexes the full mobile version. This can very well work in your favour if certain scripts or media files are being used for an immersive experience as these can be skipped on the AMP version for a higher technical score.
Google is changing search on a conceptual level with mobile-first indexing, moving from a somewhat-technical link based way of categorising information to a more evolved understanding of how various pieces of information tie in together. What we should be appreciating here is the absolute vastness of the change they are driving with this thought process.
It is also starting to get reflected on the analytics side of things with their move to Google Analytics 4 that is transforming the way we look at data (again, moving from a session/ time/ link based approach to a more holistic user experience based perspective). This changing landscape is so completely fascinating that most of our time these days is spent learning and unearthing newer concepts and strategies about the direction in which search and analytics industries are moving.
If you have not jumped the content-driven mobile-first bandwagon yet, this is the perfect time to do so. Give these above tactics a try to optimise your website for search, and reach out to us if you get stuck anywhere.
0 Comments