本文转自
http://blog.csdn.net/shenlanzifa/article/details/42679577
在运行spark的测试程序SparkPi时,点击运行,出现了如下错误:
Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.<init>(SparkContext.scala:185)
at SparkPi$.main(SparkPi.scala:12)
at SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
从提示中可以看出找不到程序运行的master,此时需要配置环境变量。
传递给spark的master url可以有如下几种:
local 本地单线程
local[K] 本地多线程(指定K个内核)
local[*] 本地多线程(指定所有可用内核)
spark://HOST:PORT 连接到指定的 Spark standalone cluster master,需要指定端口。
mesos://HOST:PORT 连接到指定的 Mesos 集群,需要指定端口。
yarn-client客户端模式 连接到 YARN 集群。需要配置 HADOOP_CONF_DIR。yarn-cluster集群模式 连接到 YARN 集群。需要配置 HADOOP_CONF_DIR。
点击菜单工具栏RUN,选择edit configuration,在左侧点击该项目。在右侧VM options中输入“-Dspark.master=local”,指示本程序本地单线程运行,再次运行即可。
配置如图:
1
相关推荐
Exception in thread "main" org.apache.spark.SparkException: Amaster URL must be set in your configuration at org.apache.spark.SparkContext.(SparkContext.scala:206) at org.apache.spark.api.java....
Bus configuration is assumed to be consisting of external device as a master (e.g. a PC) and one or more devices as slave(s). For bus-operation the device address together with baudrate must be set ...
Normally, only the columns in the WHERE clause are helpful in determining useful indexes, but for a covering index, all columns must be included. If all columns needed for the query are in the index,...
of a selected file.<END><br>7 , logging.zip This is a bas that will log installation procedures so the file can be removed later.<END><br>8 , savetree.zip This will save the info in a Tree View....
and it can also load in 640K DOS memory RDISK is a simple and small RAMdisk driver for use when resizing or other features are not needed UIDE is a DOS "Universal IDE" caching driver It...
Additional information can also be stored in a log file. A wide variety of information can be saved for future reference. Simultaneous Analysis or Defragmentation Operations ----------------------...
3. Choose a project in the Solution Explorer and set it with a right click as Startup Project. 4. Hit F5 to build and execute the sample. How to build and run the console samples on Windows, Linux and...
makeinfo is usually part of the texinfo package in your distribution make: *** [dependencies] 错误 1 make:离开目录“/home/wwt/linux_r16/lichee/buildroot” ERROR: build buildroot Failed rootroot@cm-...
This is synonymous with deliverable, that is defined as “any measurable, tangible, verifiable outcome, result or item that must be produced to complete a project or part of a project. Often used ...
you must delete it from your hard disk. Incompatible partitions are listed as "Non-DOS" partitions in FDISK. See the procedure for repartitioning hard disks in Chapter 4 of the Microsoft MS-DOS ...
depends on your network administrator to set up a DHCP server on your network. A DHCP server is scheduled to ship as part of Windows NT(TM) Server version 3.5. If you enable automatic DHCP ...
but not validated on the IResp interface resulting in a lost transaction. by the logical layer. - Version fixed : v5.1 - CR#474894 - Fixed issue when the maintenance response is followed ...
4194304000 (4000MB)Set a config from every node in cluster: $redis-cluster-tool -a 127.0.0.1:34501 -C "cluster_config_set maxmemory 10000000" -s Do you really want to execute the "cluster_config_...
This is a major release which has changed the Application Programmer Interface (API), requiring applications to be recompiled. This SoftDevice version is not Production tested on all IC revisions, and...
A moderately used database server must be configured as a NetBackup client. The database application is custom built and is not covered by a NetBackup database agent. The server is in production ...
--enable-vmx=2 option (x86-64 must be enabled) - Bugfixes for CPU emulation correctness - Fixed Bochs crash when accessing the first byte above emulated memory size - Internal Debugger - ...
"Java for the Web with Servlets, JSP, and EJB: A Developer’s Guide to J2EE Solutions" by Budi Kurniawan is a comprehensive guide aimed at helping developers master web programming in Java. The book ...
device such as a total station or theodolite in combination with a stadia rod or drop line to measure water surface elevations (Harrelson, et ai., 1994, Western et ai., 1997). Inaccuracies in ...
It provides a robust set of components that developers can use to create dynamic and interactive pages. These components include: - **Standard Components**: These are pre-built UI elements like text...