i have created an external table in hive, let's say employees. This employees table has 3 partitions, let'say country, age, and gender. Am using my own custom input format and serde implementations for this external table. While executing select queries against this external table, am getting the following exception.
java.lang.RuntimeException: java.lang.IllegalArgumentException: Can not create a Path from a null string at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.lang.IllegalArgumentException: Can not create a Path from a null string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:78) at org.apache.hadoop.fs.Path.(Path.java:90) at org.apache.hadoop.hive.ql.exec.MapOperator.cleanUpInputFileChangedOp(MapOperator.java:482) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1372) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:509) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
Also, have checked that "deserialize" method in the serde implementation is not getting called during the execution of select query and looks like it fails even before calling deserialize method.
Can anyone of you help here? Thanks in advance.
-Sankar
Here is the create table hql. CREATE EXTERNAL TABLE logs(id INT,class STRING,type STRING,name STRING,status STRING,duration STRING,record STRING,user STRING,ref STRING,age STRING) PARTITIONED BY(loc STRING, env STRING, hour STRING, grp STRING) ROW FORMAT SERDE 'com.main.hive.TxnSerDe' STORED AS INPUTFORMAT 'com.main.hive.TxnInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION '/tmp/logs'; After the table got created, i have altered the table for partitions as it is external table. ALTER TABLE logs ADD IF NOT EXISTS PARTITION(loc='hx',env='prd',hour='2014010220',grp='vertical') LOCATION 'hx/prd/2014010220/vertical';
i have created an external table in hive, let's say employees. This employees table has 3 partitions, let'say country, age, and gender. Am using my own custom input format and serde implementations for this external table. While executing select queries against this external table, am getting the following exception.
java.lang.RuntimeException: java.lang.IllegalArgumentException: Can not create a Path from a null string at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.lang.IllegalArgumentException: Can not create a Path from a null string at org.apache.hadoop.fs.Path.checkPathArg(Path.java:78) at org.apache.hadoop.fs.Path.(Path.java:90) at org.apache.hadoop.hive.ql.exec.MapOperator.cleanUpInputFileChangedOp(MapOperator.java:482) at org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1372) at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:509) at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
Also, have checked that "deserialize" method in the serde implementation is not getting called during the execution of select query and looks like it fails even before calling deserialize method.
Can anyone of you help here? Thanks in advance.
-Sankar
Here is the create table hql. CREATE EXTERNAL TABLE logs(id INT,class STRING,type STRING,name STRING,status STRING,duration STRING,record STRING,user STRING,ref STRING,age STRING) PARTITIONED BY(loc STRING, env STRING, hour STRING, grp STRING) ROW FORMAT SERDE 'com.main.hive.TxnSerDe' STORED AS INPUTFORMAT 'com.main.hive.TxnInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION '/tmp/logs'; After the table got created, i have altered the table for partitions as it is external table. ALTER TABLE logs ADD IF NOT EXISTS PARTITION(loc='hx',env='prd',hour='2014010220',grp='vertical') LOCATION 'hx/prd/2014010220/vertical';
0 commentaires:
Enregistrer un commentaire