Meilleur endroit pour json Serde JAR dans CDH Hadoop pour utilisation avec ruche/Hue/MapReduce - Stack Overflow

I'm using Hive/Hue/MapReduce with a json Serde. To get this working I have copied the json_serde.jar to several lib directories on every cluster node:

• /opt/cloudera/parcels/CDH/lib/hive/lib


• /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib


• /opt/cloudera/parcels/CDH/lib/hadoop/lib


• /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/lib


• ...

On every CDH update of the cluster I have to do that again. Is there a more elegant way where the distribution of the Serde in the cluster would be automatic and resistant to updates?

It depends on the version of Hue and if using Beeswax or HiveServer2:

• Beeswax: there is a workaround with the HIVE_AUX_JARS_PATH https://issues.cloudera.org/browse/HUE-1127




• HiveServer2 supports a hive.aux.jars.path property in the hive-site.xml. HiveServer2 does not support a .hiverc and Hue is looking at providing an equivalent at some point: https://issues.cloudera.org/browse/HUE-1066

I'm using Hive/Hue/MapReduce with a json Serde. To get this working I have copied the json_serde.jar to several lib directories on every cluster node:

• /opt/cloudera/parcels/CDH/lib/hive/lib


• /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/lib


• /opt/cloudera/parcels/CDH/lib/hadoop/lib


• /opt/cloudera/parcels/CDH/lib/hadoop-0.20-mapreduce/lib


• ...

On every CDH update of the cluster I have to do that again. Is there a more elegant way where the distribution of the Serde in the cluster would be automatic and resistant to updates?

It depends on the version of Hue and if using Beeswax or HiveServer2:

• Beeswax: there is a workaround with the HIVE_AUX_JARS_PATH https://issues.cloudera.org/browse/HUE-1127




• HiveServer2 supports a hive.aux.jars.path property in the hive-site.xml. HiveServer2 does not support a .hiverc and Hue is looking at providing an equivalent at some point: https://issues.cloudera.org/browse/HUE-1066

Source