Have you ever wondered where exactly you can get a 46 inch LED Samsung TV for the lowest price? Did it change last week, last month? How much did it go down by? (have you ever heard of it going up?!!)
These are questions that are of interest to people shopping online for such products but it appears that what these products retail for in various parts of the globe and within the US, various states and cities is of greater interest to the manufacturers of these products also!
Real-time prices on the web for such products help companies adjust their margins and in electronic products like LCD and LED TVs, it's a relentless push downwards anyway! With margins getting squeezed, real-time price intelligence like this is invaluable to them, it appears!
Extracting these prices automatically from various web sites periodically and keeping the backend database updated is a chore anyway!!
About ten years ago, we had a contract with a site called C2B Technologies that was building a shopping engine (like Shopping.com or Nextag.com today). C2B was subsequently bought and folded into Inktomi. Our contract was to write Java drivers that can access the public website and extract Product, Photo and Price information and stuff the store and product URLs along with all this information into an Oracle database. Then when you were shopping for something, it just queried this database and gave you an ordered list of stores and links for the product, just like what Shopping.com does.
I had a very interesting chat with Christian Koestler, CEO of Lixto. Lixto is a Austrian company headquartered in Vienna, Austria but Christian spends a lot of time in Silicon Valley since a lot of his customers are in the U.S.
Lixto's approach is very innovative I thought. It works just like some Interactive Demo creators where you pull up a software client or an interface on the web and click through a whole set of screens. The Demo Software creates a script and records the whole session while you are talking. This becomes the interactive demo.
Lixto does something similar. You just start Lixto (this is a SaaS, cloud-based software - so nothing local to install), and go through the web site, clicking on products and price information, while Lixto creates a script for automatic extraction later on. Something like this:
Once the script is recorded, it can be scheduled to run periodically and you can get your reports like this:
Very interesting product! I asked Christian about how many sites have a Captcha to test for a human being logging in instead of a computer. He claims to have some workarounds for that also.
I knew how difficult it is to keep Java drivers updated because online websites keep changing all the time and everytime it changes, you will have to change your code. Seems like that problem may have been solved to some extent!
Real-time changes in prices, especially in highly competitive consumer products are a challenge to companies. Lixto seems to have a solution for it!
Information is not Knowledge - Albert Einstein
Is the more you search online for a flight,the more the price will rise.Is this true?
Yes. Airline tickets do use what is called Dynamic Pricing to adjust the price of tickets upwards or downwards based on demand on certain flight sectors, days and times. For example, Friday evening flights are always in demand from business folks returning back home from their travels. That's why Saturdays and Sundays take sudden dips in prices for the same flights.
I am not sure that anyone has the technology to track which sites you have been to and what you have been searching for although they are nearing that with Behavioral Targeting at least with ads that remember that you visited Toyota's website. When you then go to CNN.com or some other completely unrelated site, the Toyota ad may pop up!
I guess it's only a matter of time before that happens!
I am looking to set up a system that continuously provides background and data to a portable.Something that allows a continuous flow of intelligence on either companies or competitive products. What systems have people implemented using open source or free tools out there.
I am unable to give him any practical advantage. Can any one give me some real life scenario that I discuss him in order to give him more insight and convince him to use Data warehousing.
Friday evening flights are always in demand from business folks returning back home from their travels.
Something that allows a continuous flow of intelligence on either companies or competitive products. What systems have people implemented using open source or free tools out there.