Batch scraping with a data table input

Now we have learned how to declare arguments in the recipe, and then execute batch scraping by inputing multiple argument lines on the recipe start UI.

Sometime, we would like to tell recipe to read these arguments from a data table. For example,

  • we have a offline csv file with all argument values
  • one recipe scraped data to an output table, and we would like another recipe to read these data and continue to do deep scraping.

NDS supports to accept data table as a recipe's input. However, before that, we have to tell recipe the mapping between input table's column and recipe's arguments. We can finish all these configuration with 2 steps:

  • Step 1: declare a data table as recipe's input

    add a 'Open Table as Argument Input' action at the beginning of Start Node

    select the table to be used as input.

    Now we have declared that the recipe accept the table as input.

  • Step 2: link input table's column to an argument In the last section, we can define an argument by type a name and default value. Here to link the argument to the input table's column, we need to select an column for the argument. Now when executing the recipe, NDS will read each row from the input data table, and assign the column to linked argument.

    For example, in the screenshot, we select one column '关键词' from the input table and link it to the argument.

Note: once declaring data table as arugment input, all old arguments will be cleaned, and you have to map columns in the input table to argument input via Argument Setting UI again.