impala on yarn 配置
- 作者: 广州的段友互粉吧
- 来源: 51数据库
- 2020-10-04
Spark On Yarn配置很简单，不需要单独部署Spark集群，只需要开发Spark应用程序，打成jar包，然后使用spark-submit命令提交到Yarn上运行即可。示例：
./spark-submit?--class?com.lxw1234.test.WordCount?--master?yarn-cluster?--executor-memory?4G?--num-executors?10?/home/lxw1234/spark-wordcount.jar?/logs/2015-07-14/?/tmp/lxw1234/output/你可以搜索一下"lxw的大数据田地"，里面有很多Spark On Yarn的技术文章。
export hadoop_conf_dir=/etc/hadoop/conf
./spark-submit --class com.lxw1234.test.wordcount --master yarn-cluster --executor-memory 4g --num-executors 10 /home/lxw1234/spark-wordcount.jar /logs/2015-07-14/ /tmp/lxw1234/output/
具体可搜索"lxw的大数据田地"，里面有spark on yarn的详细介绍。
./spark-submit?--class?com.lxw1234.test.WordCount?--master?yarn-cluster?--executor-memory?4G?--num-executors?10?/home/lxw1234/spark-wordcount.jar?/logs/2015-07-14/?/tmp/lxw1234/output/你可以搜索一下"lxw的大数据田地"，里面有很多Spark On Yarn的技术文章。
export hadoop_conf_dir=/etc/hadoop/conf
./spark-submit --class com.lxw1234.test.wordcount --master yarn-cluster --executor-memory 4g --num-executors 10 /home/lxw1234/spark-wordcount.jar /logs/2015-07-14/ /tmp/lxw1234/output/
具体可搜索"lxw的大数据田地"，里面有spark on yarn的详细介绍。
推荐阅读
