Resolving problems with running Apache Nutch on Windows. Part 2.

When Nutch crawling process started with '-solr' argument you can face a problem with Solr schema.xml.Some fields can be absent and you will see errors in log:

2012-01-25 11:03:46,484 WARN  mapred.LocalJobRunner - job_local_0020
org.apache.solr.common.SolrException: ERROR: [doc=http://business.ngs.ru/] unknown field 'content'

ERROR: [doc=http://business.ngs.ru/] unknown field 'content'

request: http://localhost:8983/solr/update?wt=javabin&version=2
    at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:436)
    at org.apache.solr.client.solrj.impl.CommonsHttpSolrServer.request(CommonsHttpSolrServer.java:245)
    at org.apache.solr.client.solrj.request.AbstractUpdateRequest.process(AbstractUpdateRequest.java:105)
    at org.apache.solr.client.solrj.SolrServer.add(SolrServer.java:49)
    at org.apache.nutch.indexer.solr.SolrWriter.close(SolrWriter.java:93)
    at org.apache.nutch.indexer.IndexerOutputFormat$1.close(IndexerOutputFormat.java:48)
    at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:474)
    at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:411)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:216)

To fix this I've copied schema.xml for Solr from http://svn.apache.org/viewvc/nutch/branches/branch-1.4/conf/schema.xml?view=markup.

Resolving problems with running Apache Nutch on Windows. Part 1.

First root of some problems with running Apache Nutch on Windows is Cygwin, as Nutch has bash scripts to run. So, install Cygwin and add its root directory to 'path' system variable (e.g. C:\Cygwin). But if you added only root directory to path you can see exception in Nutch log:

java.io.IOException: Expecting a line not the end of stream
    at org.apache.hadoop.fs.DF.parseExecResult(DF.java:109)
    at org.apache.hadoop.util.Shell.runCommand(Shell.java:179)
    at org.apache.hadoop.util.Shell.run(Shell.java:134)
    at org.apache.hadoop.fs.DF.getAvailable(DF.java:73)
    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathForWrite(LocalDirAllocator.java:329)
    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathForWrite(LocalDirAllocator.java:124)
    at org.apache.hadoop.mapred.MapOutputFile.getSpillFileForWrite(MapOutputFile.java:107)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1221)
    at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1129)
    at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:359)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)

To resolve this you should add Cygwin 'bin' directory to your system path (e.g. C:\Cygwin\bin)