As you are reading this, chances are your facing the google indexing issue like many other bloggers throughout the world, that too not only on blogger.com but wordpress, tumbler etc. Well Google is the main search engine as we all know and all of us depend on it for our traffic. It not only keeps us happy by sending readers to our blogs, but pays us for those readers through it's programs like adsense. So, what happens when such issues arise. I'm quite regular at Google Help Forums, and keep on looking at the current issues, so here is our detailed analysis.
Occurrences in History
Last year that is 2012, in the month of october, there was same problem, when most of the bloggers were facing fall in traffic after 19th of October, as the fresh content was added with delay or not added at all. At that time, JohnMu (Google Employee and Forum Moderator) appeared on the Google Help forum for webmasters and officially announced the issues that were going on.
Soon Google fixed the Indexing problem and things were restored to normal.
Today (Feb 2013), we are not sure, whether this is the same problem or it's something else, but the symptoms resemblance, points towards the same direction. There have been no official announcement from Google on this topic yet. The Google Webmasters Help Forum have been flooded with the queries from bloggers across the globe, complaining the issues with Google indexing problem.
Technically, we can suggest you few things which can help you to verify, that you are not having any issues from you side. I suggest that you go through all these points carefully and only than post your query on:
Google Webmasters help Forum >> Crawling and Indexing.
In case you've already posted your query, you are advised still to go through them and update your post, if you find any issue on your side, so that contributors can help you accordingly.
1. Problems with your Robots.txt
Robots.txt is the file that helps you to allow or disallow search engines from indexing a page or part of website or complete website. In blogger blogs, you can access your robots.txt file at address like this:
http://tech.geekupd8.com/robots.txt. To modify the settings, you can visit your blogger account and visit settings and change accordingly.
Note: You can copy the robots.txt fie of Geek Upd8 and use it for yourself, as it is working perfectly. Don't forget to modify the url's accordingly.
2. Problems with .htaccess file
This file is not visible on servers. Bloggers don't have access to this file at all. The /htaccess file that is not configured properly can be a real danger for the proper indexing of the website.
3. Meta Tags
These play a great role in the websites and blogs indexing. In case you don't know much about, that means you are missing alot. Make sure you have not made the following settings in your meta tags.
<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>
These settings as can be seen direct search engines not to index the specific page as well as not the follow the links in it.
4. Server Issues
Server issues are mostly not in our hands unless you are not managing your own servers, which is not the thing in most of our cases. Sometimes servers downtime can be cause of the non indexing or the delay in indexing of the content. You can confirm this with your webhost in case you are running blog on platforms like wordpress or tumbler. Bloggers who are using blogger.com don't need to worry about it, as there have been no such downtime in blogger that could be cause of this Google Indexing problem.
5.Page Rank
This is another point that can cause the delay in your content indexing by Google. In case you don't have a good page rank, you may not get indexed that quickly. You can consider page rank as a priority order for Google spiders to Index content present on internet, just like the sitemaps priority that you fix.
6. Problems with Sitemap
Sitemap is something that plays the most vital role in the indexing of a blog or website. Once google spiders reach according their priority to your website, they are directed by your sitemap about how to go through your blog. Thus sitemaps play really vital role in indexing.
You can submit your sitemap to Google in the webmaster tools. Do it right now, if you haven't yet.
7. Search Engine Penalty
Last but not the least, it is the factor that cannot be ignored at all. Google is a great search engine. We like it cause of the quick and best results that we get from it. They are accurate. But why are they accurate. Why Google gets us the websites that actually have the content, that we are looking for. Ever wondered why ? This is because Google have it's own mechanism of selecting websites that are to stay on Google SERP's and sieve out the rest. This removal of websites from Google Search Engine Result Pages is due to the Search Engine penalties, Google in this case. The penalties can be triggered by various issues in your website, it can any of the following:
a. Low qualiy content
b. Lots of links (link farming)
c. Excessive use of keywords
d. Links from spammy websites
e. Sites generating spam
f. Sites publishing duplicate content
h. Sites that are using abusive/offensive language
These are some of the factors that trigger the search engine penalties. In case you are doing any of these, that means you are on a wrong track.
Current Issue
As you might be already knowing that you are victim of Google Indexing issues, which thousands of fellow bloggers are facing too, so stay cool and don't worry. But, i would suggest you to confirm that if this is the issue in your case. To make sure, you are a part of the same boat, point your browser to your google.com, now here, add this search query to the search box "site:geekupd8.com". In your case, replace geekupd8.com with your site url. Now, there you might be able to Search Tools, select that and select Last 24 hrs, instead of Any Time, now in case you have posted any article in last 24 hours, it should be appearing in search results page. But if there is no result found, you can conclude that you too are facing the google indexing problem.
Last year that is 2012, in the month of october, there was same problem, when most of the bloggers were facing fall in traffic after 19th of October, as the fresh content was added with delay or not added at all. At that time, JohnMu (Google Employee and Forum Moderator) appeared on the Google Help forum for webmasters and officially announced the issues that were going on.
Today (Feb 2013), we are not sure, whether this is the same problem or it's something else, but the symptoms resemblance, points towards the same direction. There have been no official announcement from Google on this topic yet. The Google Webmasters Help Forum have been flooded with the queries from bloggers across the globe, complaining the issues with Google indexing problem.
What can i could do right now ?
Perhaps, this is the time when we you to wait and watch, and give sometime to officials at Google and see why this is being caused.Technically, we can suggest you few things which can help you to verify, that you are not having any issues from you side. I suggest that you go through all these points carefully and only than post your query on:
Google Webmasters help Forum >> Crawling and Indexing.
In case you've already posted your query, you are advised still to go through them and update your post, if you find any issue on your side, so that contributors can help you accordingly.
1. Problems with your Robots.txt
Robots.txt is the file that helps you to allow or disallow search engines from indexing a page or part of website or complete website. In blogger blogs, you can access your robots.txt file at address like this:
http://tech.geekupd8.com/robots.txt. To modify the settings, you can visit your blogger account and visit settings and change accordingly.
Note: You can copy the robots.txt fie of Geek Upd8 and use it for yourself, as it is working perfectly. Don't forget to modify the url's accordingly.
2. Problems with .htaccess file
This file is not visible on servers. Bloggers don't have access to this file at all. The /htaccess file that is not configured properly can be a real danger for the proper indexing of the website.
3. Meta Tags
These play a great role in the websites and blogs indexing. In case you don't know much about, that means you are missing alot. Make sure you have not made the following settings in your meta tags.
<META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”>
These settings as can be seen direct search engines not to index the specific page as well as not the follow the links in it.
4. Server Issues
Server issues are mostly not in our hands unless you are not managing your own servers, which is not the thing in most of our cases. Sometimes servers downtime can be cause of the non indexing or the delay in indexing of the content. You can confirm this with your webhost in case you are running blog on platforms like wordpress or tumbler. Bloggers who are using blogger.com don't need to worry about it, as there have been no such downtime in blogger that could be cause of this Google Indexing problem.
5.Page Rank
This is another point that can cause the delay in your content indexing by Google. In case you don't have a good page rank, you may not get indexed that quickly. You can consider page rank as a priority order for Google spiders to Index content present on internet, just like the sitemaps priority that you fix.
6. Problems with Sitemap
Sitemap is something that plays the most vital role in the indexing of a blog or website. Once google spiders reach according their priority to your website, they are directed by your sitemap about how to go through your blog. Thus sitemaps play really vital role in indexing.
You can submit your sitemap to Google in the webmaster tools. Do it right now, if you haven't yet.
7. Search Engine Penalty
Last but not the least, it is the factor that cannot be ignored at all. Google is a great search engine. We like it cause of the quick and best results that we get from it. They are accurate. But why are they accurate. Why Google gets us the websites that actually have the content, that we are looking for. Ever wondered why ? This is because Google have it's own mechanism of selecting websites that are to stay on Google SERP's and sieve out the rest. This removal of websites from Google Search Engine Result Pages is due to the Search Engine penalties, Google in this case. The penalties can be triggered by various issues in your website, it can any of the following:
a. Low qualiy content
b. Lots of links (link farming)
c. Excessive use of keywords
d. Links from spammy websites
e. Sites generating spam
f. Sites publishing duplicate content
h. Sites that are using abusive/offensive language
These are some of the factors that trigger the search engine penalties. In case you are doing any of these, that means you are on a wrong track.
Google Indexing Problem : New Posts are not getting Indexed in usual time, Countless #Bloggers Effected, posted on GeekUpd8 - Technology Blog by Advocate Puneet Batish - http://g8.geekupd8.com/GoogleIndexingProblem
ReplyDeletethanks for information, i was so much worried on seeing my stats, new posts were not getting indexed and i was feeling like it's over. i hope google will fix this soon. no one answered to my query in the google help forum. thanks anyways
ReplyDeleteHi my blog url is www.gotricky.com. my blog posts were getting indexed very quickly earlier, now they are taking long time like 4 days. please help. i complained in google forum, but no one helped me. please help. Ashwin
ReplyDeleteIt seems like all Google cares about is being faster than Yahoo and Bing, after that their efforts are wasted. I have noticed this - I just thought it was something I was doing wrong on my blog.
ReplyDeleteI am also facing same problem so i have to use google fetch feature, i cant see any update in total crawled and indexed url in sitemap
ReplyDelete@dhmz2012
ReplyDeleteThis problem is really big, you can have a look at the google webmasters forum. it's flooded with complaints.
Suggestion : Keep updating your blog with fresh content. If this issue was an algorithmic change intentionally introduced by Google, your fresh content will help you regain your earlier status. otherwise too it will get indexed with time.
hello,my site is new and i got indexed for the first two three days but not nothing and i stumbled across your site..i think my settings are correct..i hope it gets fixed..as a new site its like cutting my wings off..
ReplyDelete