Thursday, April 12

Amazon CloudSearch, Elastic Search as a Service

The search division of Amazon, A9 today announced the release of CloudSearch.  Amazon CTO Werner Vogels announced it on his blog, All Thing Distributed.  The AWS service also has a new post on the announcement.

For the details and pricing, there is also the official CloudSearch details page.

CloudSearch is a fully managed search service based on Amazon's search infrastructure that provides near-realtime, faceted, scalable search.  The index is stored in memory for fast search and updates.

Dynamic Scaling
What makes A9 offering particularly interesting is it's ability to dynamically scale.  The architecture of A9's search system, with shards and replicas, is a common and well-understood model.  What makes Amazon's offering unique is the ability to easily scale your search cluster.  A9 will automatically add (and remove) search instances and index partitions as the index size grows and shrinks.  It will also dynamically add and remove replicas to respond to changes in search request traffic.    The exact details are still not clearly described technically in detail.

Right now, there is a limit to 50 search instances.  An extra large search instance can handle approximately 8 Million 1K documents. It appears that assumption is that the documents are quite small (e.g. product documents).  To put it in perspective, an rough rule of thumb for web documents is approximately 10k.  Given this, it translates into roughly 800k web documents per server * 50 servers = 40 million web documents.  This is not for building large-scale web search, yet.  However, it should be more than enough for most enterprise e-commerce and site-search applications.

The real value added by the search engine is in the ranking of results.

Ranking
The control over the search index ranking is rudimentary with a few basic knobs.  You can add stopwords, perform stemming, and add synonyms.  This is very basic stuff.    How you might do more interesting (and important) IR ranking changes is vague.  From the article,
Rank expressions are mathematical functions that you can use to change how search results are ranked. By default, documents are ranked by a text relevance score that takes into account the proximity of the search terms and the frequency of those terms within a document. You can use rank expressions to include other factors in the ranking. For example, if you have a numeric field in your domain called 'popularity,' you can define a rank expression that combines popularity with the default text relevance score to rank relevant popular documents higher in your search results.
This indicators that it is possible to boost documents.  However, it is unclear how the underlying text search works in order to boost individual important fields (e.g. name, description).

For more details on the more advanced query processing needed to make search work in practice, read the post: Query Rewriting in Search Engines from Hugh Williams at EBay.  In order to employ these methods, you need log data, which brings me to my next point.

Missing Pieces
A key missing component is usage-driven framework to improve ranking that uses queries, clicks, and other user behavior indicators.  A feedback mechanism to change ranking based analysis (ideally automatic).

Overall, the most compelling aspect of this is the dynamic scaling.  It gives people a simple, platform that scales transparently for many enterprise search and ecommerce applications.


71 comments:

  1. This announcement is great but disappointing.
    A search service was the last missing piece to build a scalable web product easily on AWS so it's great to get one.
    However, I was really expecting something built on Solr for 2 reasons.
    - It will have been an easy replacement for a lot of existing users
    - Configuration and extension will have been much more powerful.

    Let's hope that this will come in the future, maybe when Solr 4 will be finalized.

    ReplyDelete
  2. I agree that API incompatibility with existing systems such as Solr will reduce the incentive to experiment with this service. As far as scalability, it's not clear to me that most enterprise search is so bursty as to need significant back-end scalability. Maybe I am overlooking something.

    What enterprise search definitely needs is a way to handle lots of document formats seamlessly. I would love a service that told me what part of my PDF or Word document matched the query in a way that I could highlight that in the UI.

    ReplyDelete
  3. Many knowledge and information I got from your site, thanks and greetings of friendship from Indonesia.

    ReplyDelete
  4. I was very pleased to find this site.I wanted to thank you for this great read!

    ReplyDelete
  5. Nowadays, most of the businesses rely on cloud based CRM tool to power their business process. They want to access the business from anywhere and anytime. In such scenarios, salesforce CRM will ensure massive advantage to the business owners. Salesforce Training | Salesforce Training in Chennai

    ReplyDelete
  6. Excellent post!!! In this competitive market, customer relationship management plays a significant role in determining a business success. That too, cloud based CRM product offer more flexibility to business owners to main strong relationship with the consumers. Salesforce Training Institutes in Chennai | Salesforce Training in Chennai

    ReplyDelete
  7. this is very nice article from the author , and dont forget to visit my special article at obat paten

    ReplyDelete
  8. Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    cloud computing training in chennai | cloud computing courses in chennai

    ReplyDelete
  9. Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    salesforce training in chennai | salesforce training institute in chennai

    ReplyDelete
  10. Thanks for sharing this informative content which provided me the required information about the latest technology.
    Salesforce training in Chennai | Salesforce course in Chennai

    ReplyDelete
  11. Posts shared useful information and meaningful life, I'm glad to be reading this article and hope to soon learn the next article.
    Justhost Coupons

    ReplyDelete
  12. Thank you for this valuable information. Get your business to the next level in simple steps. erp in chennai | erp for automotive industry chennai

    ReplyDelete

  13. very useful info, and please keep updating........
    Best Online Software Training

    ReplyDelete
  14. I will do niche blog comment Just in 5$ .All comment relevant with your niche and UNIQUE .This off-page seo will increase your traffic and promote your business.

    ReplyDelete
  15. Excellent post. I used to be checking continuously this blog and I’m impressed! Jelly Gamat Bali
    Very helpful info particularly the ultimate part
    care for such information much. I used to be seeking this
    particular info for a long time.

    ReplyDelete
  16. It is designed with the intention to keep your Computer system safe from threats, virus, and malware. It is automated protection which aims to provide complete online security.
    Bitdefender customer support

    ReplyDelete
  17. ACER customer support provides the high class customer services to our precious customers. We never make our customer unsatisfied with our services. We have team of experts who take care of their customer queries 24*7 and make our clients happy with complete satisfaction. Acer Tech Support

    ReplyDelete
  18. Dell Customer care Number provide the support to their user if any one is having some problem then he or she can share your problems here.
    Dell Customer Support Number

    ReplyDelete
  19. Excellent read, Positive site, where did u come up with the information on this posting? I have read a few of the articles on your website now, and I really like your style. Thanks a million and please keep up the effective work.

    ReplyDelete
  20. Which post you have shared on your blog that is really nice and informative I like your post thanks for the sharing.
    lenovo phone number

    ReplyDelete
  21. Hi! This is my first visit to your blog! We are a collection of volunteers and Your blog provided us valuable information to work on. You have done a outstanding job! Yahoo Customer Care

    ReplyDelete
  22. amazing post and thanks for share this

    ReplyDelete
  23. It is a very informative post. It really helps us a lot.
    AOL Customer Support Number

    ReplyDelete
  24. Technical support may be delivered over by phone or a tool where users can log a call or incident. Larger organizations frequently have internal technical support available to their staff for computer-related problems.
    Windows tech support

    ReplyDelete
  25. nice post I enjoyed it a lot keep sharing like this Hp printer Support

    ReplyDelete
  26. Thanks for sharing a nice article really such a wonderful site you have done a great job once more thanks a lot.

    ReplyDelete
  27. Thanks for sharing informative article on cloud computing technology. Your article helped me a lot in understand the future of cloud technology. Having strong expertise in leading cloud based CRM like Salesforce will ensure better career prospects for aspiring professionals. Salesforce Training in Chennai |salesforce developer training in chennai|salesforce administrator training in chennai

    ReplyDelete
  28. Thanks a lot sir for sharing such a precious tips. It relay help lot. And I feel proud to share these tips.
    Chrome Customer Support Number

    ReplyDelete
  29. I really enjoyed while reading your article, the information you have delivered in this post was damn good. Keep sharing your post with efficient news.
    Regards,
    cloud computing training in chennai|cloud computing training

    ReplyDelete
  30. Thanks a lot sir for sharing such a precious tips. It relay help lot. And I feel proud to share these tips.
    Chrome Customer Support Number

    ReplyDelete
  31. Sap Training Institute in Noida-Webtrackker is the exceptional SAP training center in noida. SAP or systems software and products in facts processing are an extraordinary a part of ERP or organization aid planning. ERP is an integration of several software or packages this is used to streamline the techniques of a big scale enterprise or business. ERP has been essential parts of enterprise organization way manipulate and one of the principal structures of ERP has been SAP.
    Sas Training Institute in Noida
    PHP Training Institute in Noida
    Hadoop Training Institute in Noida
    Oracle Training Institute in Noida
    Linux Training Institute in Noida
    Dot net Training Institute in Noida
    Salesforce training institute in noida
    Java training institute in noida

    ReplyDelete
  32. This is an interesting article about hair and its different style.
    Canon Phone Number

    ReplyDelete
  33. This post is very nice I really like to read about it.
    Canon Support

    ReplyDelete
  34. Australia best tutor is well known academic portal. Here students can get different kind of Online Assignment help services like that

    1.Online Assignment Help
    2.Instant Assignment Help
    3.Assignment Help
    4.Online Assignment Help
    5.my assignment Help

    And also access that services at any time and any where.

    ReplyDelete
  35. The article you have shared here is very awesome. I really like and appreciate your work. The points you have mentioned in this article are useful. I must try to follow these points and also share others.
    Hifi Escorts In Delhi

    ReplyDelete
  36. very Help Full Post. Look Some Different here at NDA exam

    ReplyDelete
  37. Window ac repair Birmingham-only repair center is the best fact that Microsoft is stopping support for XP and transferring their safety specialists to the later running structures is certainly an awesome signal for Windows XP users, in a manner.

    1. Air Conditioning Installation birmingham,
    2. Commercial air conditioning repair Birmingham,
    3. residential air conditioning repair birmingham ,
    4. Air Conditioning Maintenance birmingham ,
    5. window ac repair birmingham,

    ReplyDelete
  38. Good blog and informative content..Thank you for updating...VLSI Project Center in Chennai | VLSI Project Center in Velachery

    ReplyDelete
  39. Thanks for sharing this information. I really like your blog post very much. You have really shared a informative and interesting blog post with people.
    Escorts in Gurgaon

    ReplyDelete
  40. Nice post. it was effective and interesting. Thanks for sharing and keep on updating.
    Best AWS Training Institute in Chennai | Best AWS Training Institute in Velachery

    ReplyDelete
  41. Nice post.Thanks for sharing! I enjoyed reading it;The information was very useful.Keep the good work going on!
    Best cloud Computing Training Institute in Chennai | Best cloud Computing Training Institute in Velachery

    ReplyDelete
  42. CIITN Noida provides Best java training in noida based on current industry standards that helps attendees to secure placements in their dream jobs at MNCs.The curriculum of our Java training institute in Noida is designed in a way to make sure that our students are not just able to understand the important concepts of the programming language but are also able to apply the knowledge in a practical way.

    Java is inescapable, stopping meters, open transportation passes, ATMs, charge cards and TVs wherever Java is utilized.
    What's more, that is the reason Well-prepared, profoundly gifted Java experts are high sought after.

    If you wanna best java training, java industrial training, java summer training, core java training in noida, then join CIITN Noida.

    ReplyDelete
  43. Thank you for your post. This is excellent information. It is amazing and wonderful to visit your site.

    Best Linux training in Noida
    Linux Training Institute in Noida
    Shell Scripting Training Institute in Noida

    ReplyDelete
  44. تعد شركة تركيب اثاث ايكيا بالرياض هي الشركة الرائده والاولي في كافة الاثاث من تركيب وفك ونقل وتخزين وكافة الاعمال المتعلقة بالاثاث في الرياض وكافة المناطق والمحافظات بالمملكة العربية السعودية، وقد تصدرت شركة خبراء المملكة لتكون الأولى في مجال فك ونقل وتركيب الأثاث المنزلي وايضا فك وتركيب الستائر بالرياض وهي تتميز عن باقي شركات الرياض نظرا لما تقدمة من خدمات بشكل احترافي كما انها تتميز عن غيرها بكفاءة الفنيين والامتخصصين في مجال تركيب الاثاث فلا داعي لكثرة البحث فلديك خبراء المملكة فهم فعلا خبراء ومتميزون في جميع خدماتهم المقدمة
    شركة تركيب اثاث ايكيا بالرياض
    فني تركيب اثاث ايكيا بالرياض
    شركة تركيب ستائر بالرياض
    عامل تركيب ستائر بالرياض
    شركة تركيب غرف نوم بالرياض
    فني تركيب غرف نوم بالرياض
    شركة تركيب باركية بالرياض
    شركة تركيب عفش بالرياض
    ما يميز شركة تركيب نقل وتركيب اثاث بالرياض
    - تعد شركة تركيب اثاث ايكيا من الشركات المفضلة لكثير من العملاء فهم من منحوها الصدارة والتميز لتميز الخدمات المقدمة لهم وهي الاولي في تركيب الاثاث لزيادة خبراتها الكبيرة لسنوات.
    تتميز ايضا شركة خبراء المملكة بكبر فرق العمل المتخصصة والمدربة بمهاره وتقنية عالية كما اننا ندعم صفوفنا بصفة مستمرة من العمال والموظفين والفنيين الأكفاء والمهرة كما انها Jستقبل العمالة الفليبنية

    ReplyDelete
  45. Superb. I really enjoyed very much with this article here. Really it is an amazing article I had ever read. I hope it will help a lot for all. Thank you so much for this amazing posts and please keep update like this excellent article.thank you for sharing such a great blog with us. expecting for your.
    Best Web Designing Institute in Chennai
    Best web designing course in chennai
    web design classes
    PHP Training Center in Chennai
    PHP Institutes in Chennai
    PHP courses in chennai

    ReplyDelete
  46. Really amazing information!!! Thanks for your blog.

    edu-exam
    Article submission sites

    ReplyDelete