Comments on: How to create a mashup and not die trying… http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/ Random musings about nothing in particular Mon, 06 Oct 2008 21:18:52 +0000 http://wordpress.org/?v=2.6.2 By: Tim http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-1202 Tim Fri, 01 Feb 2008 01:12:39 +0000 http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-1202 “What is the best way to get information from Internet automatically?” I have had good success with the free iMacros for Firefox extension https://addons.mozilla.org/en-US/firefox/addon/3863 http://wiki.imacros.net/Data_Extraction Tim “What is the best way to get information from Internet automatically?”

I have had good success with the free iMacros for Firefox extension

https://addons.mozilla.org/en-US/firefox/addon/3863

http://wiki.imacros.net/Data_Extraction

Tim

]]> By: Chris_C http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-505 Chris_C Tue, 11 Dec 2007 06:15:18 +0000 http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-505 Nice work Franchu. We are well aware of the Mac client issue and will be addressing it. Timing is uncertain at this point. Additionally, you can create a REST robot on openkapow that can return data to you in multiple formats without rewriting the robot. Formats include: - REST (XML) - JSON - CSV - HTML - XHTML Nice work Franchu. We are well aware of the Mac client issue and will be addressing it. Timing is uncertain at this point.

Additionally, you can create a REST robot on openkapow that can return data to you in multiple formats without rewriting the robot. Formats include:
- REST (XML)
- JSON
- CSV
- HTML
- XHTML

]]> By: Franchu http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-140 Franchu Fri, 15 Jun 2007 10:23:00 +0000 http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-140 Hi! Well, I am afraid there is no easy answer to your question. In an ideal world, the page would be written in XHTML or following some accessibility standards and it would be easy to extract semantic content from the page. In the former case you would need an XML parser and using XPath you can extact the node of the document that contains the information you are interested in. In the later, you would have to parse the string, looking for the delimiters of the content you want to extract. The only thing I can do, is suggest you to search for more information on web scraping (that's the technical name of what you are trying to do). Keep in mind that the HttpClient the only thing that does is retrieve the web page and give it to you as a Stream or as a String. After this point, it is up to you to parse the content with web scrapping techniques. Hope this reply was useful to put you on track towards the answer :) Hi!

Well, I am afraid there is no easy answer to your question.

In an ideal world, the page would be written in XHTML or following some accessibility standards and it would be easy to extract semantic content from the page.

In the former case you would need an XML parser and using XPath you can extact the node of the document that contains the information you are interested in.

In the later, you would have to parse the string, looking for the delimiters of the content you want to extract.

The only thing I can do, is suggest you to search for more information on web scraping (that’s the technical name of what you are trying to do). Keep in mind that the HttpClient the only thing that does is retrieve the web page and give it to you as a Stream or as a String. After this point, it is up to you to parse the content with web scrapping techniques.

Hope this reply was useful to put you on track towards the answer :)

]]> By: discoverall http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-139 discoverall Fri, 15 Jun 2007 04:53:59 +0000 http://franchu.net/2007/06/05/how-to-create-a-mashup-and-not-die-trying/#comment-139 Hello Franchu , I asked the question "What is the best way to get information from Internet automatically?" on J2EE free course... Thank you for your witting. Now I have once more question about this. I have used HttpClient from http://jakarta.apache.org/commons/httpclient/ and I can download webpage. But I would like to get some part in that page. Example: I have a link to a news from other web. But I just want to get only the news. not get the banner, advertise... How can I do that. Thank you very much. Hello Franchu ,
I asked the question “What is the best way to get information from Internet automatically?” on J2EE free course…
Thank you for your witting.
Now I have once more question about this.
I have used HttpClient from http://jakarta.apache.org/commons/httpclient/
and I can download webpage. But I would like to get some part in that page.
Example: I have a link to a news from other web. But I just want to get only the news. not get the banner, advertise…
How can I do that.
Thank you very much.

]]>