Loading Profile...
Twitter,
Facebook, or email.
A question for all users. My WP blog is new, and I've noticed that Google indexes my blog posts very quickly, but new blog pages (which would include idx listings) much more slowly. As of this writing, not a single listing on my blog has been indexed. I checked a local competitor and fellow dsIDXpress user, and noticed that the listings indexed on his blog have a cache date nearing a month old. (i.e., no new listings indexed in the last few weeks) This experience conflicts with reviews I've read about this product where they created a brand new blog and had 1,000 pages indexed within a week.
This brings up some questions:
1.) Could it be that Google is already adjusting to this type of product, and is compensating by indexing generated pages much more slowly?
2.) I am experimenting with increasing the custom crawl rate in Google Webmaster Tools. I understand that this doesn't increase how often Googlebots crawl the site, but should increase "freshness and number of crawled pages". Has anyone experienced improved results by increasing the crawl rate, especially with a new site?
Please chime in with any experience you have regarding Google, crawling, indexing and related.
Thanks!
I’m
interested
Official
Response
-
@alex The listings are technically "posts", not pages, but google doesn't know one from the other, all they see is content. When it comes to wordpress the only real difference between a post and a page is how it's linked in the hierarchy of your site.
@intown the only adjusting google is doing is starting to realize you have a lot more content than before. The amount you get indexed relies completely on how "interesting" google thinks your site is. They're only going to spend a certain amount of time on your site indexing, so you need to make sure you've got the right caching plugins implemented (dsIDXpress tries to implement caching in all places FYI) AND (a BIG one) make sure you have a FAST FAST host so it can pull content from diverse, compute it's business, and get it to the consumer (or googlebot) asap.
@intown as for your #2, I don't have enough experience to tell you how successful you'll be, but from what I do know is: googlebot is going to do whatever the heck it wants, whether you tell it to go left or right, it might just decide to go up.
@suz I need to double check the hook flow, but I believe the "ping" you mention is fired via a post/page "publish", of which the properties don't fire, for numerous reasons, and I don't expect them to ever fire.
Here's a thought for you all: we must rely heavily on good and targeted links in your navigation, specifically to the listing areas you actually care about. Don't flood google with all the cities in your MLS, like I've seen some customers do, try to target a few Cities, or even neighborhoods, that you think you expect to be able to cater to. That way you can shape google to index the content most important to you. (this may be elementary, but I've seen many clients drop a page with all few hundred cities/zips/tracts/communities and still only have 100 odd pages indexed.
After that, build your traditional SEO stuff, I know it sounds like sunday school, but the more incoming links, relevant content, and FAST page load times, the more pages you'll get indexed on.
EMPLOYEE
Create a customer community for your own organization
Plans starting at $19/month
-
I have a good feeling google is going through some changes and how they are ranking the updates. One problem is that the sitemap is not including all the properties - just cities, etc...
Google went crazy on my site when I added the dsidxpress. But I haven't seen the hits I was expecting and don't see my site when buyers are searching mls's or property address. some properties are getting indexed but not to enough.
I suppose I can give it more time, but I'm thinking it takes 3-4 months for things to happen. In the meantime, I have to create unique content in order to generate search engine traffic and hard links to my site.
This of course takes even more time and resources. Other than paying for the traffic, I don't see anything else at this point. I do believe sites that have unique content and back-links are having the property addresses indexed quickly.Comment -
-
I’m
happy
1I agree with Richard, I think many need to be patient with the indexing! I think many have the expectation that your posts & pages are going to be magic once you create them and implement the dsIDXpress!
Content is still king, write, write and write. But write fairly good stuff. Write for the search terms for your area! Work the terms into your post and neighborhood posts & pages! Add the dsIDXpress as an 'enchancement' and create pages for cities, neighborhoods and listings!
You CAN NOT expect to have a new site to experience 1,000's of indexed pages over night! I can tell you from experience that it did not take me overnight to get prospective buyers to my site to search for homes! I've have spent hours upon hours, months upon months working on my site, learning about blogging and testing and trying new things!
When I implemented the Diverse Solutions Search Agent full product on my site, it took a good three months and a lot of posting to start getting some traffic and lead caption! Be patient!
"BUILD IT and IT WILL COME"
During that time, use the dsIDXpress to enhance your site, follow some of the suggestions here from those that have 'aged' sites and watch what they do.
Implement the site maps, learn about pinging, build community pages, utilize the widgets and short codes!! Be creative, original and please search out at least 12-15 keywords that you are trying to dominate for your area and focus on those keywords in posts, keywords, meta descriptions and keep checking back to see how you fair in the organic results (SERP's)...
Content, Patience, Content, Patience, Keywords (long tail also), Patience, test, test and test some more! Set up your Google Analytics.
It will not happen overnight - don't expect it too! -
-
I’m
just asking questions!
CRobinson, thanks for your reply, but it doesn't really answer my questions. I don't think I implied anywhere that I expected this product to be "magic", did I? I'm quite familiar with building pages for niche areas and we're getting ranked quite nicely for our targeted communities. Our main site gets 25,000 page views per month, which is pretty decent for the little mom & pop operation that we are. We're converting leads quite well with dsSearchAgent on our main site. I implemented this plugin to have indexable listings on our WP blog, IOW, to be found when people Googled an address, partial address or MLS number. Our main site brings in the major keyphrases we're targeting. Sure, my WP blog might get some of that to, but that would be an ancillary benefit to us.
So, back to my actual questions. I started by asking about the differences in speed with which Google indexes a blog post vs a page. I also wished to compare our experience to those of another who claimed to install this plugin on a brand-new blog and had 1,000 pages indexed within a week. I don't expect our pages to be #1 or even on the first page necessarily, but I just expected more pages to just be indexed period, even if they are way down in the SERPs to start.
My question #1 asked if Google may be adjusting to this type of product. I think it's a legitimate question, and I have evidence to indicate that might be true. As I mentioned in another thread (http://community.dsidxpress.com/diver...) when you Google "site:myblog/com" it apparently doesn't show every page that Google actually has indexed. While using that method it only shows 10 properties indexed, but Google Analytics shows we had visitors to 279 different property pages. So perhaps using "site:myblog/com" is not the proper metric.
My question #2 asked about increasing the crawl rate in GWT to see if that helped. Has anyone tried that, and if so, what was your experience? Early adopters of this product reported huge spikes in pages crawled on their existing WP blogs. I did not. The famous blog post where the user posted a screen shot of this spike from the GWT account is now suspect to me. How do I know that person didn't just select an increased custom crawl rate before taking that screen shot?
So, in summary, I don't expect magic, I don't expect overnight success, I'm just asking legitimate questions. Please don't read any more into what I've asked than my actual words. -
-
I’m
good
Intown Elite - my apologies if I've offended you, not my intention. Perhaps, this particular thread was not the correct place to share my 'experience you have regarding Google, crawling, indexing and related' ....
I hope my response didn't come off has being a brat, I have seen several threads created with concerns about the indexing in general.
You are correct, you didn't suggest magic, again I apologize. Perhaps if anything, someone can read the information above as some kind of help. I've seen your site, your work and probably Richard's answer above was better suited for what you were looking for.
I'm not sure there is any absolute answer on how Google works and how they choose to index, we can only go with what available tools they have. I do believe that longevity and age of a site is paramount. That's my guess for maybe part of your questions. Best, Candace -
-
I’m
good too!
No worries. I know you're trying to help. Just trying to keep the conversation focused and prevent thread creep.
-
-
I sympathize with you, intown. I wish these would get indexed faster; maybe the product would work better if these were blog posts, not blog pages? I'd love to get Diverse Solutions' take on that.
The bulk of my marketplace is foreclosure property, and often these properties are sold in 1-4 days. This product doesn't move fast enough for that, but I can start Googling foreclosures 1-2 days after they come on the market and they'll turn up on other Realtors' websites. -
-
Hmmm... these are pages, not posts? I somehow got the idea that they were posts which would ping the server. No? -
-
@alex The listings are technically "posts", not pages, but google doesn't know one from the other, all they see is content. When it comes to wordpress the only real difference between a post and a page is how it's linked in the hierarchy of your site.
@intown the only adjusting google is doing is starting to realize you have a lot more content than before. The amount you get indexed relies completely on how "interesting" google thinks your site is. They're only going to spend a certain amount of time on your site indexing, so you need to make sure you've got the right caching plugins implemented (dsIDXpress tries to implement caching in all places FYI) AND (a BIG one) make sure you have a FAST FAST host so it can pull content from diverse, compute it's business, and get it to the consumer (or googlebot) asap.
@intown as for your #2, I don't have enough experience to tell you how successful you'll be, but from what I do know is: googlebot is going to do whatever the heck it wants, whether you tell it to go left or right, it might just decide to go up.
@suz I need to double check the hook flow, but I believe the "ping" you mention is fired via a post/page "publish", of which the properties don't fire, for numerous reasons, and I don't expect them to ever fire.
Here's a thought for you all: we must rely heavily on good and targeted links in your navigation, specifically to the listing areas you actually care about. Don't flood google with all the cities in your MLS, like I've seen some customers do, try to target a few Cities, or even neighborhoods, that you think you expect to be able to cater to. That way you can shape google to index the content most important to you. (this may be elementary, but I've seen many clients drop a page with all few hundred cities/zips/tracts/communities and still only have 100 odd pages indexed.
After that, build your traditional SEO stuff, I know it sounds like sunday school, but the more incoming links, relevant content, and FAST page load times, the more pages you'll get indexed on. -
-
Jon, what are "the right caching plugins"?
-
There are so many to choose from:
http://wordpress.org/extend/plugins/w...
http://wordpress.org/extend/plugins/w...
http://wordpress.org/extend/plugins/d...
http://wordpress.org/extend/plugins/d...
http://wordpress.org/extend/plugins/h...
http://wordpress.org/extend/plugins/w...
Each one claims to be better than its predecessor. WP Super Cache seems to be the most popular, with over 1.1 million downloads, but newer plugins claim to be better. Anybody got a favorite?
thanks,
Mike -
-
-
-
-
Intown said:
"Early adopters of this product reported huge spikes in pages crawled on their existing WP blogs. I did not. The famous blog post where the user posted a screen shot of this spike from the GWT account is now suspect to me. How do I know that person didn't just select an increased custom crawl rate before taking that screen shot?"
If you are referring to the post I wrote on Geek Estate Blog (http://www.geekestateblog.com/a-sneak...) there is no need to be suspect. I don't mess with the crawl rates as I don't know if that could have some other, unintended, consequence. I can't *prove* I didn't increase crawl rates, about all I can do is tell you I didn't.
Google continues to crawl and index many pages created by dsIDXpress on my blog. BUT, please keep in mind that my blog is almost five years old, has over 1300 posts and 60 - 80K backlinks (depending on whose numbers you believe). All of that contributes a lot to how often, and how deeply, Google crawls the site. It's not just the DS plugin, it's many other factors as well.
But whether someone has a blog one day old or five years old, the simple fact is the dsIDXpress plug in gives your the ability to have crawled / indexed pages. Most other IDX solutions have zero ability in this regard.-
I did read your post, but there was also one from where someone claimed to set up a brand new blog and have 1,000 pages indexed within a week. I thought that may have been Property Indexer, but their post just claimed 279 pages indexed within a week, so I'm not sure who claimed the 1000. Anyway, it would be cool if DS could somehow make it so each listing page would ping when it was generated. One could use a plugin like MBP Ping Optimizer to prevent excess pinging.
-
-
-
-
-
I use W3 Total Cache. I ran Super Cache for a long time but see better visible speed performance and have reduced server load with W3 Total Cache.
I'm no caching expert though. Which plugin works best for anyone may (MAY) depend on settings on their host as well. (I host at RackSpace Cloud).-
Trying W3 Total Cache now. Wow, lots of settings! Ugh, yet another learning curve to tackle [sigh]. Do you host static files with a CDN provider?
-
-
-
-
-
Mike - I don't use a CDN for static files.
-
-
I’m
happy
I can attest to the faster crawl rate and better indexing. Google is picking up my posts immediately and I have more than tripled what was indexed. My rankings for two terms have shot up to page one. I can see through analytics how much traffic this is generating and it's much higher than before.
Now I still trying to figure out how to make use of those unique address searches and get someone who is looking for that property to contact me. -
-
Josette - do you have a contact form that is displayed directly on the pages the plugin is generating? I'm getting my form filled out frequently by searchers landing on pages found by specific address searches.
What helps me tremendously is my contact form includes in its email to me what page the form was sent from.
You can see an example here:
http://www.phoenixrealestateguy.com/i... -
-
I’m
happy
I have a contact form but it's on the sidebar of the blog area so the referral page is set to that. I just used yours to submit and would love if you'd send back to me what you get.
I have someone who can design a template page for how to display these pages and I'm digging around to see what ideas I can steal! I think the contact form on each page that lifts the title tag since that seems to have the housing info. Something the marketing side has said is to have as part of each page more of a reason for them to contact me for this listing like I've previewed these listings and I'm the expert in the local area.
Here is what it looks like currently:
http://bethesdabuzz.com/idx/mls-mc727... -
-
Hey Jay, what did you use for your form? I've tried 3-4 different plug-ins so far, and none of them work right.
-
-
-
-
Hey you three, don't tell my boss I told you, but there's definitely a contact form on the detail pages in version 1.1. I don't have a release date for you yet, but I wanted to at least let you know before you go spending lots of time on rigging one in there. /hides-from-justin
- view 1 more comment
-
-
So, will it have cool tracking features like Enhanced WP Contact Form or Gravity? I'm sure it will tell us the listing info, but it would be cool if it told us other pages viewed on our site, the referring site, and if a search engine, the search term used. (If it just included a link of the referring site, the search term would be in the URL.)
-
-
Sorry for the duplicate reply. Seems like something is messed up with Get Satisfaction today. It didn't acknowledge my post was accepted, so I clicked Post again. Still no acknowledgement so I refreshed and saw my posting twice. It also is not letting me edit or delete like it used to. I saw this happened to a couple of other folks on this thread: http://community.dsidxpress.com/diver...
-
-
-
-
-
-
I’m
excited
I had this inside info a bit ago - patiently waiting... can't wait! Shhhh... don't mention that to anyone!
-
-
-
-
<takes-off-diverse-solutions-employee-cap>
this is just my own personal thing to drop out there for you all, thought it might be useful:
Regarding page-indexing. I know I've mentioned Page Speed being a factor, well it's official now http://googlewebmastercentral.blogspo...
So:
1. get those caching plugins functional (i like this one http://wordpress.org/extend/plugins/w...)
2. move to another host if yours is slow
3. limit the plugins on your page
4. profit
that's it off the top of my head, feel free to drop in your own 2cents. NOW don't go asking _me_ how to get this setup, I'm just the messenger :)
</takes-off-diverse-solutions-employee-cap> -
