Normally, you probably don't need to mess around with
"NUM_SCANNER_THREADS". I'm not sure how you determine that impala is
scanning files with one thread, but you can look for
"AverageScannerThreadConcurrency"
in the query profile.
Keep in mind that if you're querying only one block (or one parquet file
which is one block), then Impala will use one scanner thread.
Thanks,
Alan
On Sat, Jan 18, 2014 at 6:06 AM, hiro wrote:
Hi,
After I set "statestore_subscriber_timeout_seconds" to 60sec,
now it's working without reconnection messages.
But, there is still something wrong.
I query with the following parameter,
impala > set NUM_SCANNER_THREADS=8;
but, impala-server still scans files with one thread.
It used to work fine.
Is there still commucation problem between statestore and impala-server ?
Do you have any idea what is going on ?
Thanks,
Hiro
2014年1月6日月曜日 21時53分41秒 UTC+9 hiro:
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].Hi,
After I set "statestore_subscriber_timeout_seconds" to 60sec,
now it's working without reconnection messages.
But, there is still something wrong.
I query with the following parameter,
impala > set NUM_SCANNER_THREADS=8;
but, impala-server still scans files with one thread.
It used to work fine.
Is there still commucation problem between statestore and impala-server ?
Do you have any idea what is going on ?
Thanks,
Hiro
2014年1月6日月曜日 21時53分41秒 UTC+9 hiro:
Hi,
I am using Impala 1.2.3 on CDH4.
I installed impala before and I have run several queries successfully.
I just started Impala again after a while and
I get errors like the following when I query.
------------------
[impalad003:21000] > select * from part where p_retailprice <= 900.18;
Query: select * from part where p_retailprice <= 900.18
ERROR: Cancelled due to unreachable impalad(s): impalad006:22000,
impalad063:22000, impalad015:22000, impalad100:22000, ...
ERROR: Invalid query handle
------------------
There seems some connection problems.
I also checked impalad.INFO in each impalad node,
it keeps reconnecting to state-store over and over and never stops as
shown below.
As far as I know, it wasn't like this before, so it might be the cause.
------------------
I0106 21:46:07.333633 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:07.333675 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:07.336510 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
I0106 21:46:17.338811 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:17.338851 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:17.339157 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
W0106 21:46:24.055603 320 Configuration.java:2227]
or[email protected]:an
attempt to override final parameter: mapreduce.job.end-
notification.max.retry.interval; Ignoring.
W0106 21:46:24.056337 320 Configuration.java:2227]
or[email protected]:an
attempt to override final parameter: mapreduce.job.end-notification.max.attempts;
Ignoring.
I0106 21:46:42.344254 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:42.344295 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:42.344761 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
I0106 21:46:52.346424 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:52.346465 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:52.347231 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
W0106 21:47:03.134521 325 Configuration.java:2227]
org[email protected]:an
attempt to override final parameter: mapreduce.job.end-
notification.max.retry.interval; Ignoring.
W0106 21:47:03.135227 325 Configuration.java:2227]
org[email protected]:an
attempt to override final parameter: mapreduce.job.end-notification.max.attempts;
Ignoring.
... (never stops)
------------------
Does anyone have any idea about this ?
Thanks in advance.
Hiro
To unsubscribe from this group and stop receiving emails from it, send
an email to [email protected].I am using Impala 1.2.3 on CDH4.
I installed impala before and I have run several queries successfully.
I just started Impala again after a while and
I get errors like the following when I query.
------------------
[impalad003:21000] > select * from part where p_retailprice <= 900.18;
Query: select * from part where p_retailprice <= 900.18
ERROR: Cancelled due to unreachable impalad(s): impalad006:22000,
impalad063:22000, impalad015:22000, impalad100:22000, ...
ERROR: Invalid query handle
------------------
There seems some connection problems.
I also checked impalad.INFO in each impalad node,
it keeps reconnecting to state-store over and over and never stops as
shown below.
As far as I know, it wasn't like this before, so it might be the cause.
------------------
I0106 21:46:07.333633 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:07.333675 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:07.336510 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
I0106 21:46:17.338811 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:17.338851 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:17.339157 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
W0106 21:46:24.055603 320 Configuration.java:2227]
or[email protected]:an
attempt to override final parameter: mapreduce.job.end-
notification.max.retry.interval; Ignoring.
W0106 21:46:24.056337 320 Configuration.java:2227]
or[email protected]:an
attempt to override final parameter: mapreduce.job.end-notification.max.attempts;
Ignoring.
I0106 21:46:42.344254 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:42.344295 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:42.344761 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
I0106 21:46:52.346424 32072 state-store-subscriber.cc:202]
impalad002:22000: Connection with state-store lost, entering recovery mode
I0106 21:46:52.346465 32072 state-store-subscriber.cc:206] Trying to
re-register with state-store, attempt: 1
I0106 21:46:52.347231 32072 state-store-subscriber.cc:210] Reconnected to
state-store. Exiting recovery mode
W0106 21:47:03.134521 325 Configuration.java:2227]
org[email protected]:an
attempt to override final parameter: mapreduce.job.end-
notification.max.retry.interval; Ignoring.
W0106 21:47:03.135227 325 Configuration.java:2227]
org[email protected]:an
attempt to override final parameter: mapreduce.job.end-notification.max.attempts;
Ignoring.
... (never stops)
------------------
Does anyone have any idea about this ?
Thanks in advance.
Hiro
To unsubscribe from this group and stop receiving emails from it, send
