Need help with XML Query Parser example in Lucene 3.0

Dear fellow Java developers:

I am trying to run the XML Query Parser example that comes with Lucene 3.0
source distribution. I have the application structured identical to the
one you download, and I am trying to run it in eclipse. When I launch the
application, fill out the form that appears on index.jsp, and then submit
it to search the index, I get the following error 505:

javax.servlet.ServletException: Error processing query

org.apache.lucene.xmlparser.webdemo.FormBasedXmlQueryDemo.doPost(FormBasedXmlQueryDemo.java:124)
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)


root cause

java.lang.NullPointerException

org.apache.lucene.xmlparser.webdemo.FormBasedXmlQueryDemo.doPost(FormBasedXmlQueryDemo.java:107)
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)


I absolutely have no idea why this is happening, as I even structured the
package of the servlet in eclipse identical to the way it is in the
example. I have attached my entire servlet, "FormBasedXmlQueryDemo"
below:

public class FormBasedXmlQueryDemo extends HttpServlet {
private static final long serialVersionUID = 1L;

private QueryTemplateManager queryTemplateManager;
private CorePlusExtensionsParser xmlParser;
private IndexSearcher searcher;
private Analyzer analyzer = new
StandardAnalyzer(org.apache.lucene.util.Version.LUCENE_CURRENT);

public FormBasedXmlQueryDemo() {
super();
// TODO Auto-generated constructor stub
}

@Override
public void init(ServletConfig config) throws ServletException {
// TODO Auto-generated method stub
super.init(config);

try{

openExampleIndex();

String xslFile=config.getInitParameter("xslFile");
String defaultStandardQueryParserField =
config.getInitParameter("defaultStandardQueryParserField");
queryTemplateManager = new
QueryTemplateManager(getServletContext().getResourceAsStream("/WEB-INF/" +
xslFile));
xmlParser = new
CorePlusExtensionsParser(defaultStandardQueryParserField, analyzer);

}catch(Exception e){
throw new ServletException("Error loading query template", e);
}
}

protected void doGet(HttpServletRequest request, HttpServletResponse
response) throws ServletException, IOException {

}

@Override
protected void doPost(HttpServletRequest request, HttpServletResponse
response) throws ServletException, IOException {
// TODO Auto-generated method stub

Properties completedFormFields = new Properties();
Enumeration pNames = request.getParameterNames();

while (pNames.hasMoreElements()){

String propName = (String) pNames.nextElement();
String value = request.getParameter(propName);

if ((value != null) && (value.trim().length()>0)){
completedFormFields.setProperty(propName, value);
}
}

try{
org.w3c.dom.Document xmlQuery =
queryTemplateManager.getQueryAsDOM(completedFormFields);
Query query = xmlParser.getQuery(xmlQuery.getDocumentElement());
TopDocs topDocs = searcher.search(query, 10);

if (topDocs != null) {
ScoreDoc[] sd = topDocs.scoreDocs;
Document[] results = new Document[sd.length];

for(int i = 0; i < results.length; i++){

results[i] = searcher.doc(sd[i].doc);
request.setAttribute("results", results);
}
}

RequestDispatcher dispatcher =
getServletContext().getRequestDispatcher("/index.jsp");
dispatcher.forward(request, response);
}
catch(Exception e){
throw new ServletException("Error processing query", e);
}
}

private void openExampleIndex() throws CorruptIndexException, IOException
{


RAMDirectory rd = new RAMDirectory();
IndexWriter writer = new IndexWriter(rd, analyzer,
IndexWriter.MaxFieldLength.LIMITED);
InputStream dataIn =
getServletContext().getResourceAsStream("/WEB-INF/data.tsv");
BufferedReader br = new BufferedReader(new InputStreamReader(dataIn));
String line = br.readLine();

while (line != null){

line = line.trim();
if (line.length() > 0){

StringTokenizer st = new StringTokenizer(line, "\t");
Document doc = new Document();
doc.add(new Field("location",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("salary",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("type",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("description",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED));
writer.addDocument(doc);
}

line = br.readLine();
}

writer.close();
}

}



When I was tracing through the code, it appears that it was throwing the
exception just as it tried to process the line:

TopDocs topDocs = searcher.search(query, 10);

Does anyone know where I am going wrong?

Thanks in advance to all who reply.

Dear fellow Java developers:

I am trying to run the XML Query Parser example that comes with Lucene 3.0
source distribution. I have the application structured identical to the
one you download, and I am trying to run it in eclipse. When I launch the
application, fill out the form that appears on index.jsp, and then submit
it to search the index, I get the following error 505:

javax.servlet.ServletException: Error processing query

org.apache.lucene.xmlparser.webdemo.FormBasedXmlQueryDemo.doPost(FormBasedXmlQueryDemo.java:124)
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
javax.servlet.http.HttpServlet..service(HttpServlet.java:803)


root cause

java.lang..NullPointerException

org.apache.lucene.xmlparser.webdemo.FormBasedXmlQueryDemo.doPost(FormBasedXmlQueryDemo.java:107)
javax.servlet.http.HttpServlet.service(HttpServlet.java:710)
javax.servlet.http.HttpServlet.service(HttpServlet.java:803)


I absolutely have no idea why this is happening, as I even structured the
package of the servlet in eclipse identical to the way it is in the
example. I have attached my entire servlet, "FormBasedXmlQueryDemo"
below:

public class FormBasedXmlQueryDemo extends HttpServlet {
private static final long serialVersionUID = 1L;

private QueryTemplateManager queryTemplateManager;
private CorePlusExtensionsParser xmlParser;
private IndexSearcher searcher;
private Analyzer analyzer = new
StandardAnalyzer(org.apache.lucene.util.Version.LUCENE_CURRENT);

public FormBasedXmlQueryDemo() {
super();
// TODO Auto-generated constructor stub
}

@Override
public void init(ServletConfig config) throws ServletException {
// TODO Auto-generated method stub
super.init(config);

try{

openExampleIndex();

String xslFile=config.getInitParameter("xslFile");
String defaultStandardQueryParserField =
config.getInitParameter("defaultStandardQueryParserField");
queryTemplateManager = new
QueryTemplateManager(getServletContext().getResourceAsStream("/WEB-INF/" +
xslFile));
xmlParser = new
CorePlusExtensionsParser(defaultStandardQueryParserField, analyzer);

}catch(Exception e){
throw new ServletException("Error loading query template", e);
}
}

protected void doGet(HttpServletRequest request, HttpServletResponse
response) throws ServletException, IOException {

}

@Override
protected void doPost(HttpServletRequest request, HttpServletResponse
response) throws ServletException, IOException {
// TODO Auto-generated method stub

Properties completedFormFields = new Properties();
Enumeration pNames = request.getParameterNames();

while (pNames.hasMoreElements()){

String propName = (String) pNames.nextElement();
String value = request.getParameter(propName);

if ((value != null) && (value.trim().length()>0)){
completedFormFields.setProperty(propName, value);
}
}

try{
org.w3c.dom..Document xmlQuery =
queryTemplateManager.getQueryAsDOM(completedFormFields);
Query query = xmlParser.getQuery(xmlQuery.getDocumentElement());
TopDocs topDocs = searcher.search(query, 10);

if (topDocs != null) {
ScoreDoc[] sd = topDocs.scoreDocs;
Document[] results = new Document[sd.length];

for(int i = 0; i < results.length; i++){

results[i] = searcher.doc(sd[i].doc);
request.setAttribute("results", results);
}
}

RequestDispatcher dispatcher =
getServletContext().getRequestDispatcher("/index..jsp");
dispatcher.forward(request, response);
}
catch(Exception e){
throw new ServletException("Error processing query", e);
}
}

private void openExampleIndex() throws CorruptIndexException, IOException
{


RAMDirectory rd = new RAMDirectory();
IndexWriter writer = new IndexWriter(rd, analyzer,
IndexWriter.MaxFieldLength.LIMITED);
InputStream dataIn =
getServletContext().getResourceAsStream("/WEB-INF/data.tsv");
BufferedReader br = new BufferedReader(new InputStreamReader(dataIn));
String line = br.readLine();

while (line != null){

line = line.trim();
if (line.length() > 0){

StringTokenizer st = new StringTokenizer(line, "\t");
Document doc = new Document();
doc.add(new Field("location",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("salary",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("type",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED_NO_NORMS));
doc.add(new Field("description",st.nextToken(),Field.Store.YES,
Field.Index.ANALYZED));
writer.addDocument(doc);
}

line = br.readLine();
}

writer.close();
}

}



When I was tracing through the code, it appears that it was throwing the
exception just as it tried to process the line:

TopDocs topDocs = searcher.search(query, 10);

Does anyone know where I am going wrong?

Thanks in advance to all who reply.

Hi Fayyaz,
Good job! I found an error in your code so that makes us even :)

It looks like you removed the line in the "openExampleIndex" method which
opens the searcher.
That explains your null pointer.

The problem you found in the web.xml isn't actually an error but is
potentially misleading. In the "UserQuery" tag in the XSL there is a
"fieldName" tag which is set to "description". The "jobDescription"
default fieldname passed to the XML parser would only be in effect for any
<UserQuery> tags that didn't specify a fieldName..

BTW, in the source distribution there are full "DTDdocs" for the XML
syntax in contrib\xml-query-parser\docs

Cheers
Mark


----- Original Message ----
From: syedfa <fayyazuddin@gmail.com>
To: java-user@lucene.apache.org
Sent: Wed, 23 December, 2009 5:03:00
Subject: Re: Need help with XML Query Parser example in Lucene 3.0


I have found an error in the web.xml file, however, this DID NOT fix the
problem. Inside the web.xml file, there is the following snippet:

<init-param>
<description>
Default field used in standard Lucene QueryParser used in
UserQuery
tag</description>
<param-name>defaultStandardQueryParserField</param-name>
<param-value>jobDescription</param-value>
</init-param>

the <param-value> here should be IMHO "description", as neither the
query.xsl file, nor index.jsp file contain any tags/values called
"jobDescription", only "description". I hope the contributors to the
Lucene
project take note of this, or please show me the error of my ways. Having
said that, as I mentioned earlier, despite making this correction, I am
still getting the above "NullPointerException". Can anyone see where I am
going wrong?

Thanks again to all who reply.
Sincerely;
Fayyaz

syedfa wrote:
--
View this message in context:
http://old.nabble.com/Need-help-with-XML-Query-Parser-example-in-Lucene-3.0-tp26896711p26897607.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org





---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Thanks very much for your kind reply, and for pointing out my mistake. I
made the correction, (I can't believe I left that line out!) This was a
total oversight on my part. Having said that, after making the change, I
re-ran the application, but now I'm getting no results to appear. I've
tried entering "java" for the job description field, and I've tried
entering "lucene" also, while arbitrarily choosing "permanent", and
selecting the various locations (north, south, east, west). Any reason
why that would be? I walked through the code, and found that in the
following block:

TopDocs topDocs = searcher.search(query, 10);
if (topDocs != null) {
ScoreDoc[] sd = topDocs.scoreDocs;
Document[] results = new Document[sd.length];

for(int i = 0; i < results.length; i++){

results[i] = searcher.doc(sd[i].doc);
request.setAttribute("results", results);
}
}


In the above code block, when I create my TopDocs object, using the
debugger in eclipse, I see that the value of totalHits is 0, and scoreDocs
has a value of ScoreDoc[0]. Because of this, my Document[] results array
is created with a zero index, and therefore nothing gets returned to the
user. I can't see why this is, since I'm using "java", "lucene", "HTML"
as keywords in the search field.

Can you see where I'm going wrong?

Thanks again for your help. I truly appreciate it.

Fayyaz


markharw00d wrote: