如何配置hive on spark

zhaopingzi

为什么我配置了hive on spark,查询时还走mr

1. 
   
2. hive> set hive.execution.engines=spark;
   
3. hive> select * from gprs_service_bdr12345 where start_time>='2014-07-09 10:00:08' and start_time<'2014-10-24 10:24:10.8' and lac= 10408  limit 10;
   
4. Total jobs = 1
   
5. Launching Job 1 out of 1
   
6. Number of reduce tasks is set to 0 since there's no reduce operator
   
7. Starting Job = job_1438159351661_0082, Tracking URL = http://node1:18088/proxy/application_1438159351661_0082/
   
8. Kill Command = /home/hadoop/hadoop-2.2.0/bin/hadoop job  -kill job_1438159351661_0082
   
9. Hadoop job information for Stage-1: number of mappers: 8; number of reducers: 0
   
10. 2015-08-06 09:54:48,109 Stage-1 map = 0%,  reduce = 0%
    
11. 2015-08-06 09:54:56,426 Stage-1 map = 88%,  reduce = 0%, Cumulative CPU 25.99 sec
    
12. 2015-08-06 09:54:57,473 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 31.08 sec
    
13. MapReduce Total cumulative CPU time: 31 seconds 80 msec
    
14. Ended Job = job_1438159351661_0082
    
15. MapReduce Jobs Launched:
    
16. Job 0: Map: 8   Cumulative CPU: 31.08 sec   HDFS Read: 10148488 HDFS Write: 27934 SUCCESS
    
17. Total MapReduce CPU Time Spent: 31 seconds 80 msec
    
18. OK
    
19. 2014-07-09 10:02:53.562 0       -636472190      8613702644929   460002647377121 183298258       0       0       0131820083020411        460     0       10408   255   4681     0       0       0       -575583661      -575578822      -575583661      -575578810      cmnet   2       30728   56      7       1       592     80      4.625 0.625    100000  2014-07-09 10:02:53.562 2014-07-09 10:02:54.562 4       62807   6       2026164842      80      0       0               1       86      200     756   -575583661       other   271     10675   13      0       0       2
    

复制代码

wenhq

能说下为什么不需要走mr？应该是yarn-submit之类的提交给spark吧？