`
mandwuyee
  • 浏览: 35436 次
  • 性别: Icon_minigender_1
  • 来自: 武汉
社区版块
存档分类
最新评论

[转]A master URL must be set in your configuration错误

阅读更多

本文转自

http://blog.csdn.net/shenlanzifa/article/details/42679577

 

在运行spark的测试程序SparkPi时,点击运行,出现了如下错误:

Exception in thread "main" org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.<init>(SparkContext.scala:185)
at SparkPi$.main(SparkPi.scala:12)
at SparkPi.main(SparkPi.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties

 

从提示中可以看出找不到程序运行的master,此时需要配置环境变量。

传递给spark的master url可以有如下几种:

local 本地单线程
local[K] 本地多线程(指定K个内核)
local[*] 本地多线程(指定所有可用内核)
spark://HOST:PORT 连接到指定的 Spark standalone cluster master,需要指定端口。
mesos://HOST:PORT 连接到指定的 Mesos 集群,需要指定端口。
yarn-client客户端模式 连接到 YARN 集群。需要配置 HADOOP_CONF_DIR。

yarn-cluster集群模式 连接到 YARN 集群。需要配置 HADOOP_CONF_DIR。

点击菜单工具栏RUN,选择edit configuration,在左侧点击该项目。在右侧VM options中输入“-Dspark.master=local”,指示本程序本地单线程运行,再次运行即可。

配置如图:

1

分享到:
评论

相关推荐

    Spark源码....

    Exception in thread "main" org.apache.spark.SparkException: Amaster URL must be set in your configuration at org.apache.spark.SparkContext.(SparkContext.scala:206) at org.apache.spark.api.java....

    IFC110 485模块

    Bus configuration is assumed to be consisting of external device as a master (e.g. a PC) and one or more devices as slave(s). For bus-operation the device address together with baudrate must be set ...

    微软内部资料-SQL性能优化5

    Normally, only the columns in the WHERE clause are helpful in determining useful indexes, but for a covering index, all columns must be included. If all columns needed for the query are in the index,...

    VB编程资源大全(英文源码 其它)

    of a selected file.&lt;END&gt;&lt;br&gt;7 , logging.zip This is a bas that will log installation procedures so the file can be removed later.&lt;END&gt;&lt;br&gt;8 , savetree.zip This will save the info in a Tree View....

    uhdd.sys源码

    and it can also load in 640K DOS memory RDISK is a simple and small RAMdisk driver for use when resizing or other features are not needed UIDE is a DOS &quot;Universal IDE&quot; caching driver It...

    Diskeeper 2008 v12.0.759.0

    Additional information can also be stored in a log file. A wide variety of information can be saved for future reference. Simultaneous Analysis or Defragmentation Operations ----------------------...

    OPC UA 客户端 服务器 标准库源码

    3. Choose a project in the Solution Explorer and set it with a right click as Startup Project. 4. Hit F5 to build and execute the sample. How to build and run the console samples on Windows, Linux and...

    linux全志R16的linux系统编译的资料_20170502_1655.7z

    makeinfo is usually part of the texinfo package in your distribution make: *** [dependencies] 错误 1 make:离开目录“/home/wwt/linux_r16/lichee/buildroot” ERROR: build buildroot Failed rootroot@cm-...

    a project model for the FreeBSD Project.7z

    This is synonymous with deliverable, that is defined as “any measurable, tangible, verifiable outcome, result or item that must be produced to complete a project or part of a project. Often used ...

    MS-DOS 5.0

    you must delete it from your hard disk. Incompatible partitions are listed as "Non-DOS" partitions in FDISK. See the procedure for repartitioning hard disks in Chapter 4 of the Microsoft MS-DOS ...

    win 3.11 for workgroup tcpip支持

    depends on your network administrator to set up a DHCP server on your network. A DHCP server is scheduled to ship as part of Windows NT(TM) Server version 3.5. If you enable automatic DHCP ...

    k7 SRIO参考例程

    but not validated on the IResp interface resulting in a lost transaction. by the logical layer. - Version fixed : v5.1 - CR#474894 - Fixed issue when the maintenance response is followed ...

    Redis集群管理工具redis-cluster-tool.zip

    4194304000 (4000MB)Set a config from every node in cluster: $redis-cluster-tool -a 127.0.0.1:34501 -C "cluster_config_set maxmemory 10000000" -s Do you really want to execute the "cluster_config_...

    nRF51822 softdevice s110_nrf51_8.0.0

    This is a major release which has changed the Application Programmer Interface (API), requiring applications to be recompiled. This SoftDevice version is not Production tested on all IC revisions, and...

    killtest 免费提供Symantec 251-622考试题库下载

    A moderately used database server must be configured as a NetBackup client. The database application is custom built and is not covered by a NetBackup database agent. The server is in production ...

    Bochs - The cross platform IA-32 (x86) emulator

    --enable-vmx=2 option (x86-64 must be enabled) - Bugfixes for CPU emulation correctness - Fixed Bochs crash when accessing the first byte above emulated memory size - Internal Debugger - ...

    Java_for_the_Web_with_Servlets

    "Java for the Web with Servlets, JSP, and EJB: A Developer’s Guide to J2EE Solutions" by Budi Kurniawan is a comprehensive guide aimed at helping developers master web programming in Java. The book ...

    雷达技术知识

    device such as a total station or theodolite in combination with a stadia rod or drop line to measure water surface elevations (Harrelson, et ai., 1994, Western et ai., 1997). Inaccuracies in ...

    salesforce dev 501 Notes

    It provides a robust set of components that developers can use to create dynamic and interactive pages. These components include: - **Standard Components**: These are pre-built UI elements like text...

Global site tag (gtag.js) - Google Analytics