suggested this on December 16, 2011, 8:01 AM
Campaign crawls for HTTPS is now working, anyway we can get this for Open Site Explorer?
Yes! This is in testing right now and by the end of next week, a small amount of HTTPS sites will start being crawled in Open Site Explorer. Because of the nature of our index, it will take a few index updates for us to crawl and find a higher percentage of the HTTPS sites that exist because we have to find other sites linking to them in order to discover them.
Thanks everyone on this thread for your patience! This has been a much bigger task than we anticipated, and we're really excited to get it out for you all.
Miranda Rensch - Product Manager
Hi there! Our current ETA for https sites in OSE is four weeks from now. We plan to begin crawling https sites when the next index kicks off next week, and then if all goes well with the index it will be there 2-3 weeks after the index starts.
We apologize for the continued wait. This is something our team has been working on very hard for the last couple of months; thank you so much for your patience.
Is functionality with the crawlers turned on yet? Our OSE data is virtually useless right now.
Here is the latest status of where we are in regards to HTTPS sites being crawled and indexed in Open Site Explorer:
1. We're dev complete - huge step!
2. Friday the team walked through a really positive round of testing to make sure the the behaviors in Open Site Explorer were as expected
3. Next step: final testing and then turning on functionality in our crawlers.
I hate to keep saying "it's almost there", but we're really close at this point. As soon as we have the functionality turned on within our crawlers, I'll update this post and let you all know an ETA of when to expect to see HTTPS data within the index. Once the functionality is turned on within our crawlers, it will take about two to three weeks before it's processed in our index. The first index that does contain HTTPS data will be launched with a corresponding blog post as an announcement.
I know you are all eager to see this functionality, as are we! I really appreciate your patience as we get closer to being ready for release!
Agreed... Any update? I've been waiting for this for almost a year already. As a paying customer with an HTTPS website, it's been pretty frustrating to say the least, not to have accurate data with which to make decisions.
Hi Carin, any news on this?
Hey folks! Wanted to update this forum post with the latest information regarding including HTTPS data in Open Site Explorer.
Yep! We have just started testing of this feature and hope to start seeding HTTPS sites into our crawlers early November if things continue to go well. We'll only be seeding a few HTTPS sites initially, but as our crawlers organically discover and index more HTTPS sites, they will continually be added into the index.
Once we begin indexing HTTPS sites, you'll be able to see data across all of our apps if you site is in the in the index - including Moz Analytics, Open Site Explorer and the MozBar. We'll be publishing a blog post and updating this forum post as soon we are able to release HTTPS into the Mozscape index!
Is this feature any closer to going live? Are you now at a point where you have a likely date it will be launched?
Hey Nic (and others subscribed to this thread)!
Looks like Adam has jumped in here to answer a few of the questions around adding HTTPS to Open Site Explorer, but let me fill you all in on the latest:
I hope that helps answer your questions, but let me know if you have any others. I know this is a huge feature request from a lot of our users so we're really excited to get this one released as soon as we can. We're aggressively aiming for late September or early October and will be announcing the release on the Moz blog as soon as we start releasing HTTPS data.
Would like to know when this gets rolled out. Frustrated being unable to use Moz to crawl my sites on https servers
Sorry, ignore my last post. Comments are sorted other way around, my bad.
So, let's hope to see those HTTPS stats soon, good work.
Any news in regards to this feature? Last response was about 18 months ago and I encountered the same issue as the people above as our entire website is HTTPS.
Oh wow, this is fantastic and very timely. I'm working with a company that does lots of business with government agencies and a portion of its site is secure with a good deal of links to them, but it's tough getting a big picture of who is doing all that linking. Can't wait to see this.
As I understand it, the HTTPS crawl data will work it's way into the indexes iteratively over time, and this should start in one of the next few indexes. This means that we should have better and better secure site representation over the next few months. I'll try and keep this thread posted as we see uptake with this.
With regard to the SERP analysis reports, if you are asking about metrics for https pages, this will come along for the ride, as will access via the API, campaigns, and OSE.
I hope this helps.
One more question, will HTTPS metrics also be enabled for SEOMoz PRO SERP analyses?
*meant "HTTPS option"
Hi Adam --
Does this mean OpenSiteExplorer will be supporting HTTPS websites (or an HTTS option) in its next crawl?
Hi RKG-Dev -
When we release this you will be able to retrieve https URLs and associated metrics from the API in the same way you currently retrieve http metrics. We will be crawling https URLs at the same time we crawl http URLs so they will be available on the same dates.
Will this be available in the API as well? Will it be on the same dates?
Really hoping for this feature too!
Hi Sophie -We have the majority of the engineering work on this completed. I expect we'll be able to release this feature in the second quarter of this year.
I am eagerly waiting to work this feature in OSE. Can you please share some updates in this regards?
Hello there! Sorry about the wait. It turned out to be a more complicated change than we had expected originally. We just added the type of testing framework that we'll need to test this type of addition, so the addition should be in place by March of next year. Again, I appoligize for the wait. Thank you!
Miranda Rensch - Product Manager
Any updates on this? It's been on the list and "a couple of months away" for years now.
Sadly, this is still a ways out. This will require a significant change to Linkscape, and we are focused on some larger improvements to the scale and speed of the index first. Rest assured that this isn't forgotten and is still an important project in our backlog.
Any ways to get Open Sie Explorer to crawl HTTPS sites?
Thanks for the suggestion! This is on our list of important product updates, but is still at least a few months out. Unfortunately it is a much more significant problem to solve than it was for campaigns. I'll try and keep this thread posted as we come closer as we begin work on this project.
Support Software by Zendesk