Results 1 to 6 of 6
Like Tree5Likes
  • 3 Post By James Watt
  • 1 Post By Eric Rohrback
  • 1 Post By Eric Rohrback

Thread: SEMRush Bot seen on results page

  1. #1
    Ana
    Ana is offline Member
    Member Since
    Apr 2015
    Location
    UK
    Posts
    10
    Thanks
    1
    Thanked 0 Times

    SEMRush Bot seen on results page

    Hi Guys

    Recently, I've a SEMRush BOT appear on my server search results, instantly placed a #SEO block then after a number of days unblocked it. Question is, why are they here?

    Thanks
    Advanced reply Reply  

  2. #2
    James Watt's Avatar
    James Watt is offline Top Contributor
    Member Since
    Oct 2013
    Location
    Portland, OR
    Posts
    821
    Thanks
    139
    Thanked 296 Times

    Re: SEMRush Bot seen on results page

    When you say you saw it in the server search results, you mean in your log files on your host? Is your question, just wondering why SEMRush was crawling your site?

    There's a huge number of bots out there, apparently over half of all internet traffic is from bots. Most friendly bots crawl your site for the same general reason humans do... to find information. I don't know about SEMRush, but a common reason for bots to crawl might be to index pages, or find the link structure connecting your site with other sites. SEMRush likely isn't hurting anything, but (unless you're using their service) they probably aren't helping anything either. Your data's just being used to help others make informed marketing decisions, if you care enough to block, there's no downside for you at least. Unless you have reason to believe the bots are using up more than their fair share of server load though, there's not really any reason to take the time to block them.
    Advanced reply Reply  

  3. #3
    Ana
    Ana is offline Member
    Member Since
    Apr 2015
    Location
    UK
    Posts
    10
    Thanks
    1
    Thanked 0 Times

    Re: SEMRush Bot seen on results LOG files

    I think I know very good SEO. I actually taught myself via the white hat method, but have poor CSS, so I won't show my website. Yes in the log files. I did some reading on their
    HTML Code:
    https://www.semrush.com/bot/
    and others then decided afterwards to re-block them in my /robots.txt as basically they are criminals pinching other people's hard work for their Adsense benefit. Someone stated a long while ago: Thou shalt not steal.

    i believe the reason they are here is because of my high ranking for some top quality keywords i an using.

    Thanks
    Advanced reply Reply  

  4. #4
    Eric Rohrback's Avatar
    Eric Rohrback is offline Global Moderator
    Member Since
    Oct 2012
    Location
    Pittsburgh, PA
    Posts
    917
    Thanks
    132
    Thanked 249 Times

    Re: SEMRush Bot seen on results LOG files

    They are visiting your site to document for their tools, not adsense. SEMrush is a reporting software that helps monitor keyword performance for organic ranking and paid search. They are one of the better companies out there to help understand site performance.

    I've used both SEMrush and SpyFu, and never had a problem with them. If you block all bots from your site, then hopefully you have developed some internal/proprietary software for your own use. Otherwise you're going to have a tough time understanding the full market landscape.
    djbaxter likes this.
    My rarely updated website (I should fix that) - https://www.ericrohrback.com
    Follow me on Twitter
    Want to talk? Book time with me here
    Advanced reply Reply  

  5. Members who thanked Eric Rohrback for this post:

    djbaxter (10-23-2017)

  6. #5
    Ana
    Ana is offline Member
    Member Since
    Apr 2015
    Location
    UK
    Posts
    10
    Thanks
    1
    Thanked 0 Times

    Re: SEMRush Bot seen on results page

    Eric, they are here like you say and because I have mastered (this area) how to fully optimize rank on page 1 of Google and Bing for some seriously juicy keywords and they want all the cake, but I have blocked the Semrushbot and SemRushbot-SA plus a few others just in case.
    Last edited by Ana; 10-23-2017 at 11:10 AM. Reason: ()
    Advanced reply Reply  

  7. #6
    Eric Rohrback's Avatar
    Eric Rohrback is offline Global Moderator
    Member Since
    Oct 2012
    Location
    Pittsburgh, PA
    Posts
    917
    Thanks
    132
    Thanked 249 Times

    Re: SEMRush Bot seen on results page

    @Ana - Nice trolling. We'll consider this a case closed subject since you obviously are set on what you want to do and do not want to engage in any real discussion.

    Please contribute in a meaningful way to threads in the future.
    djbaxter likes this.
    My rarely updated website (I should fix that) - https://www.ericrohrback.com
    Follow me on Twitter
    Want to talk? Book time with me here
    Advanced reply Reply  

Similar Threads

  1. SEMRush-like Ranking Report Tool for Local Results
    By nerboda in forum Local SEO Tools & Software
    Replies: 5
    Last Post: 10-28-2015, 06:11 AM
  2. Replies: 2
    Last Post: 10-21-2015, 08:04 AM
  3. Detached Local SEO - First Page Organic, 11th Page Map Results
    By marianorodriguez9 in forum Help & Support for Google Local
    Replies: 16
    Last Post: 07-14-2014, 02:59 PM
  4. Semrush
    By deanPlumberseo in forum Local SEO Tools & Software
    Replies: 5
    Last Post: 10-29-2013, 12:35 PM
  5. 2 Page 1 listings vs blended results
    By Marc in forum Local Search
    Replies: 5
    Last Post: 01-05-2013, 02:08 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •