Fork and Exec Using on JAVA
Recently I am reading the codes which write by my others. I think it will cause us problem in the future. It is like a small bomb on our server.
The system is using scala codes as follow to execute shell script
Logger.debug(String.format("Executing script: %s %s %s", "/bin/sh", shellScript, date))
val process = Runtime.getRuntime().exec(Array("/bin/sh", shellScript, date))
process.waitFor()
Logger.debug("Script execution finished, process output.")
val buf = new StringBuffer
val infos = Source.fromInputStream(process.getInputStream())(Codec.UTF8).getLines
val errors = Source.fromInputStream(process.getErrorStream())(Codec.UTF8).getLines
if (!infos.isEmpty) {
buf.append("Info:<br/>")
infos.foreach(buf.append(" ").append(_).append("<br/>"))
}
if (!errors.isEmpty) {
buf.append("Error:<br/>")
errors.foreach(buf.append(" ").append(_).append("<br/>"))
}
buf.toString()
Then it will call java.lang.Runtime.java, I am reading the codes from oracle jdk1.7 from my understanding.
public Process exec(String[] cmdarray, String[] envp, File dir)
throws IOException {
return new ProcessBuilder(cmdarray)
.environment(envp)
.directory(dir)
.start();
}
It will call the method java.lang.ProcessBuilder.java
return ProcessImpl.start(cmdarray,
environment,
dir,
redirects,
redirectErrorStream);
Open the file java.lang.ProcessImpl.java
return new UNIXProcess
(toCString(cmdarray[0]),
argBlock, args.length,
envBlock, envc[0],
toCString(dir),
std_fds,
redirectErrorStream);
Then Open the file is java.lang.UNIXProcess.java, it is the native codes.
private native int forkAndExec(int mode, byte[] helperpath,
byte[] prog,
byte[] argBlock, int argc,
byte[] envBlock, int envc,
byte[] dir,
int[] fds,
boolean redirectErrorStream)
throws IOException;
Then I check the man book on linux system from here
http://linux.die.net/man/2/fork
From the document, it is saying that
Under Linux, fork() is implemented using copy-on-write pages, so the only penalty that it incurs is the time and memory required to duplicate the parent's page tables, and to create a unique task structure for the child.
That means if the parent application is running in 4G memory, this sub process will use 4G too.
Actually our situation is worse
-Xmx6144m -Xmn2048m -XX:PermSize=192m -XX:MaxPermSize=192m
How to check the free memory on my machine
http://blog.scoutapp.com/articles/2010/10/06/determining-free-memory-on-linux
free -m total used free shared buffers cached Mem: 8008 7327 680 0 153 690 -/+ buffers/cache: 6483 1524 Swap: 0 0 0
The actually free memory will be Free(680 MB) + Buffers ( 153 MB) + cached (690 MB) = 1523 MB
On that shell script, we are using shell script only doing the curl(actually it is wget), zip (compress file), ftp related operations. So we have 2 options.
1. Do not use Shell Script
Using Open Source FTP Client http://commons.apache.org/proper/commons-net/
I read the source codes, it is using socket to deal with ftp protocol, there is no shell script there.
http://svn.apache.org/repos/asf/commons/proper/net/trunk/src/main/java/org/apache/commons/net/ftp/FTPClient.java
Using Open Source API to compress the file
http://commons.apache.org/proper/commons-compress/
2. Using a small operation provide restAPI to execute shell command
A standalone app with little memory.
References:
http://blog.csdn.net/vernonzheng/article/details/8644936
http://linux.die.net/man/2/fork
ftp client
http://commons.apache.org/proper/commons-net/
compress
http://commons.apache.org/proper/commons-compress/
- 浏览: 2551962 次
- 性别:
- 来自: 成都
文章分类
最新评论
-
nation:
你好,在部署Mesos+Spark的运行环境时,出现一个现象, ...
Spark(4)Deal with Mesos -
sillycat:
AMAZON Relatedhttps://www.godad ...
AMAZON API Gateway(2)Client Side SSL with NGINX -
sillycat:
sudo usermod -aG docker ec2-use ...
Docker and VirtualBox(1)Set up Shared Disk for Virtual Box -
sillycat:
Every Half an Hour30 * * * * /u ...
Build Home NAS(3)Data Redundancy -
sillycat:
3 List the Cron Job I Have>c ...
Build Home NAS(3)Data Redundancy
发表评论
-
Update Site will come soon
2021-06-02 04:10 1678I am still keep notes my tech n ... -
Stop Update Here
2020-04-28 09:00 316I will stop update here, and mo ... -
NodeJS12 and Zlib
2020-04-01 07:44 476NodeJS12 and Zlib It works as ... -
Docker Swarm 2020(2)Docker Swarm and Portainer
2020-03-31 23:18 369Docker Swarm 2020(2)Docker Swar ... -
Docker Swarm 2020(1)Simply Install and Use Swarm
2020-03-31 07:58 370Docker Swarm 2020(1)Simply Inst ... -
Traefik 2020(1)Introduction and Installation
2020-03-29 13:52 337Traefik 2020(1)Introduction and ... -
Portainer 2020(4)Deploy Nginx and Others
2020-03-20 12:06 431Portainer 2020(4)Deploy Nginx a ... -
Private Registry 2020(1)No auth in registry Nginx AUTH for UI
2020-03-18 00:56 436Private Registry 2020(1)No auth ... -
Docker Compose 2020(1)Installation and Basic
2020-03-15 08:10 374Docker Compose 2020(1)Installat ... -
VPN Server 2020(2)Docker on CentOS in Ubuntu
2020-03-02 08:04 455VPN Server 2020(2)Docker on Cen ... -
Buffer in NodeJS 12 and NodeJS 8
2020-02-25 06:43 385Buffer in NodeJS 12 and NodeJS ... -
NodeJS ENV Similar to JENV and PyENV
2020-02-25 05:14 478NodeJS ENV Similar to JENV and ... -
Prometheus HA 2020(3)AlertManager Cluster
2020-02-24 01:47 423Prometheus HA 2020(3)AlertManag ... -
Serverless with NodeJS and TencentCloud 2020(5)CRON and Settings
2020-02-24 01:46 337Serverless with NodeJS and Tenc ... -
GraphQL 2019(3)Connect to MySQL
2020-02-24 01:48 248GraphQL 2019(3)Connect to MySQL ... -
GraphQL 2019(2)GraphQL and Deploy to Tencent Cloud
2020-02-24 01:48 451GraphQL 2019(2)GraphQL and Depl ... -
GraphQL 2019(1)Apollo Basic
2020-02-19 01:36 328GraphQL 2019(1)Apollo Basic Cl ... -
Serverless with NodeJS and TencentCloud 2020(4)Multiple Handlers and Running wit
2020-02-19 01:19 314Serverless with NodeJS and Tenc ... -
Serverless with NodeJS and TencentCloud 2020(3)Build Tree and Traverse Tree
2020-02-19 01:19 318Serverless with NodeJS and Tenc ... -
Serverless with NodeJS and TencentCloud 2020(2)Trigger SCF in SCF
2020-02-19 01:18 294Serverless with NodeJS and Tenc ...
相关推荐
### Linux下Fork与Exec使用的相关知识点 #### 一、引言 - **Fork的概念**: `fork()` 是 Unix/Linux 操作系统中最杰出的功能之一,它允许一个正在运行的进程创建一个新进程,这个新进程被称为子进程。子进程几乎是...
Unix课程作业。 使用fork(), exec(), dup2(), pipe() ,open()系统调用完成与下列shell命令等价的功能。 grep –v usr < /etc/passwd | wc –l > result.txt
本文将简要回顾Java中的并发编程基础知识,介绍java.util.concurrent包提供的高级并发原语,并深入探讨Fork/Join框架及其在Java SE 7中的应用。 首先,让我们回顾一下Java中基本的并发机制。自Java早期版本起,线程...
Fork/Join框架是Java并发库中的一部分,自Java 7开始引入,它为开发者提供了一种高效的处理大规模计算任务的方法。这个框架基于分治策略,将大任务分解成若干小任务,然后并行执行这些小任务,最后再将结果合并。...
《OnJava8-Examples-3.0_soucecode_java_》是基于Java 8的一份源代码库,它对应于《Thinking in Java 5th Edition》这本书中的示例代码。这个压缩包包含了丰富的编程示例,旨在帮助读者深入理解Java 8的新特性以及...
本文将深入探讨两个重要的函数,`fork()`和`exec()`系列函数,它们在启动新进程时起着至关重要的作用。 **一、复制进程映像:fork()函数** `fork()`函数是Unix和类Unix系统(如Linux)中用于创建新进程的关键系统...
在UNIX和Linux操作系统中,进程创建和控制是通过一系列系统调用来实现的,其中最常见的是`fork`、`exec`系列以及`system`和`popen`函数。这些调用各自有不同的特性和用途,理解它们的区别对于进行系统级编程至关重要...
操作系统中的进程API主要涉及到三个关键系统调用:`fork`、`wait`和`exec`。这些调用在UNIX系统中被广泛使用,是构建多进程应用程序的基础。 **过程创建在UNIX系统中(通过系统调用)** 在UNIX系统中,创建新进程...
使用fork(),exec(),dup2(), pipe(),open()系统调用完成与下列shell命令等价的功能:grep -v usr | wc -l > result.txt
Fork/Join框架是Java7引入的一种用于并行任务执行的框架,它允许将复杂任务拆分成多个子任务,并行执行,然后通过join操作将结果聚合。Fork/Join框架特别适合处理可以递归拆分的计算密集型任务,比如大数据集的搜索...
掌握`fork()`和`exec`系列函数是理解和编写多进程Python程序的关键。在处理需要同时进行的异步任务,如后台服务、数据处理等场景时,这些工具非常有用。理解它们的工作原理和使用方法,能帮助开发者有效地利用系统...
在操作系统领域,Linux环境下,`fork`、`exec`和`wait`是三个非常重要的系统调用,它们在进程管理中扮演着核心角色。本文将深入解析这些函数的工作原理及其在实际应用中的重要性。 首先,`fork()`函数是创建新进程...
在Linux操作系统中,多进程编程是通过系统调用函数`fork()`来实现的。`fork()`函数能够创建一个新的子进程,使得子进程与父进程共享代码段,但各自拥有独立的堆栈段和数据段。当`fork()`成功执行时,它在父进程中...
给 Fork me on GitHub 换个风格.zip,A fresher "Fork me on GitHub" callout.
This concise book empowers all Java developers to master the complexity of the Java thread APIs and concurrency utilities. This knowledge aids the Java developer in writing correct and complex ...
`fork()`创建的子进程通常不会立即执行新的程序,而是通过`exec()`系列函数(如`execl()`, `execv()`, `execle()`, `execvp()`等)替换其当前的进程映像来运行新的程序。 七、示例代码 下面是一个简单的`fork()`...
使用linux来fork程序 "Fork",除了它是一個當你不停地敲入后看起來非常奇怪的單詞以外,通常是指 Unix 產生新進程的方式。由于系統調用的用法將會在其他 IPC 的文檔中出現,本文只是一個快速的,不太精确的 fork() ...
在这个"Java8集合 CompletableFuture lambda表达式 新的TimeAPI 和ForkJoin Demo包"中,我们可以深入探讨以下几个关键知识点: 1. **Lambda表达式**: Lambda表达式是Java 8的一大亮点,它简化了对匿名函数的处理...
Java7之forkjoin简介_动力节点Java学院整理,动力节点口口相传的Java黄埔军校
Oracle has two products that implement Java Platform Standard Edition (Java SE) 7: Java SE Development Kit (JDK) 7 and Java SE Runtime Environment (JRE) 7. JDK 7 is a superset of JRE 7, and contains ...