linkding

#.blogit #markdownify #scraper | Mercury Parser is the API that services like Feedbin and Reeder use to give you full content articles in your feed. It's shutting down, but Postlight has open sourced the parser and the API. I've been playing with a local install and it makes a great markdownifier. (I'll probably be updating Marky with it soon so I can switch over to https...)

5 years ago | View Shared by ttscoff

Parsers

#.blogit #chrome #chrome-extension #scraper | An interesting Chrome extension that lets you create web scrapers with point and click to specify components of a page, then download results as a CSV after scraping multiple pages. Use it on 10 pages without signup, 1000 with registration, and even more with the premium plan.

5 years ago | View Shared by ttscoff

hartator/wayback-machine-downloader

#.blogit #archive #cli #download #scraper #wayback | I had cause to need to recover an entire defunct section of a website recently. It existed in good shape on the Wayback Machine, but downloading sites isn't easy there. I searched for solutions a while, and eventually found wayback-machine-downloader. Worked like a charm.

7 years ago | View Shared by ttscoff

Jam API

#.blogit #api #scraper #tool | A service that can turn any website into a JSON API using CSS selectors.

8 years ago | View Shared by ttscoff

Instaparser

#.blogit #instapaper #readability #scraper | Paid access to Instapaper's web scraper. I'd like to incorporate this into [Marky](http://markdownrules.com), but the cost wouldn't even out. Still, one of the best tools available for finding the actual content in a web page.

8 years ago | View Shared by ttscoff

Portia

#.blogit #scraper #tool | The list of web scraping services keeps growing. This one is especially nice due to its visual builder, just click to highlight and define portions of a web page to create your own data-gatering tool.

8 years ago | View Shared by ttscoff

Parsebot

#.blogit #readability #scraper | Parsebot can extract article contents (full text, author, post date, title, photos, tags, and more) from any url. A great tool for integrating (legally, of course) content from one place into another (or scraping to your own archives). An API is available through a premium plan.

8 years ago | View Shared by ttscoff

diffbot/lib/diffbot at master · nickcen/diffbot

#diffbot #readability #ruby #scraper | Ruby gem for diffbot

10 years ago | View Shared by ttscoff

ddmnet/wgrep

#blogit #html #json #node #scraper | Another HTML/JSON parser written in Node. CSS selectors, including :contains(), which is really handy.

10 years ago | View Shared by ttscoff

jq

#blogit #cli #json #scraper | CLI query tool for JSON. Great for handling JSON responses in command line scripts.

10 years ago | View Shared by ttscoff

EricChiang/pup

#blogit #cli #html #scraper | This CLI is awesome for parsing HTML on the command line. Crawling, scraping, and grabbing elements and their contents is a breeze. Try `curl brettterpstra.com | pup h2.entry-title text{}`. Binaries available on the releases page. [Via OneThingWell.](http://onethingwell.org/post/97887596570)

10 years ago | View Shared by ttscoff

pjscrape: A web-scraping framework written in Javascript, using PhantomJS and jQuery

#cli #javascript #jquery #scraper | pjscrape is a framework for anyone who's ever wanted a command-line tool for web scraping using Javascript and jQuery. Built to run with PhantomJS, it allows you to scrape pages in a fully rendered, Javascript-enabled context from the command line, no browser required.

13 years ago | View Shared by ttscoff

LINKDING

Shared bookmarks

User

Tags