Detail page scraping

Visual recipe creator makes the recipe definition intutive and flexible. NDS simulates the steps you surf a website.

Here we take The GrandView Resturant on Yelp for example: https://www.yelp.com/biz/the-grandview-restaurant-san-jose

Step 1: open the page and decide what content to scrape.

Here we want to scrape the restaurant's name, website, phone, and address.

Step 2: start NDS, click 'Pro Scrape' on the popup window, and select 'Scrape Detail' scattfold

NDS will generate a recipe automatically with 2 nodes:

  • Start : a Transit node with an openURL action in default.
  • Detail1: a Detail Node with an empty field in default.

Step 3: locate elements on the page to scrape.

More detail about Element Finder, please see Element Finder guide

Step 4: decide whether to do deep scraping.

Here we can click one of declared fields with clickable element, no matter it is a link or not, to go to next page, and continue to scrape there; or we can go to the next node silently, and continue to scrape on the same page.

If deep scraping, we click 'Next' navigation button on the bottom, and NDS will ask what kind of node to create for the next one. Once node type selected, NDS will create a new node and switch to it immediately. If we select 'click field' as navigation way, we select the field to click, and the first element located by the field on the current page will be clicked, and the whole editor will be shown on the new loaded page automatically.

More details about Navigation, please see Navigation in detail

Step 5: Click 'Finish' button on the bottom of the editor to save the recipe.

If there are any required configuration missing, a warning dialog will tell which configuration is missing (on which node, and under which tab)

For example:

It tells that on Node 'List1':

  • under Data tab, the block element is missing; Field1's element or default value is required
  • Under Pages tab, you selected clicking Next button as pagination, but the Next button element is missing.

Here if you click 'continue', then the recipe will be saved as a draft one,which cannot be executed. All such draft recipes are started with ( ) icon on popup recipe list.

After a complete recipe defined, you can start it on popup window.