第一步：启动IntelliJ IDEA，选择Create New Project，然后选择Scala，点击下一步，输入项目名称wujiadong.spark继续下一步

第二步：导入spark-assembly-1.5.1-hadoop2.6.0.jar包

File——Project Structure——Libraries——点+号——点java——选择下载好的spark-assembly-1.5.1-hadoop2.6.0.jar包——点ok

第三步：创建WordCount类编写代码

第四步：导出jar包

依次选择“File”–> “Project Structure” –> “Artifact”，选择“+”–> “Jar” –> “From Modules with dependencies”，选择main函数，并在弹出框中选择输出jar位置，并选择“OK”。

最后依次选择“Build”–> “Build Artifact”编译生成jar包。具体如下图所示。

第五步：spark-submit提交运行

hadoop@master:~/wujiadong$ spark-submit --class wujiadong.spark.WordCount  --executor-memory 500m --total-executor-cores 2 /home/hadoop/wujiadong/wujiadong.spark.jar hdfs://master:9000/wordcount.txt 17/02/02 20:27:34 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable17/02/02 20:27:37 INFO Slf4jLogger: Slf4jLogger started17/02/02 20:27:37 INFO Remoting: Starting remoting17/02/02 20:27:37 INFO Remoting: Remoting started; listening on addresses :[akka.tcp://sparkDriver@192.168.1.131:52310]17/02/02 20:27:41 WARN MetricsSystem: Using default name DAGScheduler for source because spark.app.id is not set.17/02/02 20:27:44 INFO FileInputFormat: Total input paths to process : 117/02/02 20:27:51 INFO deprecation: mapred.tip.id is deprecated. Instead, use mapreduce.task.id17/02/02 20:27:51 INFO deprecation: mapred.task.id is deprecated. Instead, use mapreduce.task.attempt.id17/02/02 20:27:51 INFO deprecation: mapred.task.is.map is deprecated. Instead, use mapreduce.task.ismap17/02/02 20:27:51 INFO deprecation: mapred.task.partition is deprecated. Instead, use mapreduce.task.partition17/02/02 20:27:51 INFO deprecation: mapred.job.id is deprecated. Instead, use mapreduce.job.id(spark,1)(wujiadong,1)(hadoop,1)(python,1)(hello,4)17/02/02 20:27:52 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon.17/02/02 20:27:52 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports.17/02/02 20:27:52 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down.