sillycat
- 浏览: 2564520 次
- 性别:
- 来自: 成都
-
最新评论
-
nation：
你好，在部署Mesos+Spark的运行环境时，出现一个现象， ...
Spark(4)Deal with Mesos -
sillycat：
AMAZON Relatedhttps://www.godad ...
AMAZON API Gateway(2)Client Side SSL with NGINX -
sillycat：
sudo usermod -aG docker ec2-use ...
Docker and VirtualBox(1)Set up Shared Disk for Virtual Box -
sillycat：
Every Half an Hour30 * * * * /u ...
Build Home NAS(3)Data Redundancy -
sillycat：
3 List the Cron Job I Have>c ...
Build Home NAS(3)Data Redundancy
文章列表
Data Solution 2019(6)MySQL Data Source
Make sure our connection to database is good
> grant all privileges on database.* to root@‘142.xxx.xxx.xxx' identified by ‘xxxxxx';
> flush privileges;
In my Zeppelin Notebook
I can use this to load the dependencies
%spark.dep
z.load("mysql:mysql-con ...
Jfrog Artifactory 2019 Cluster in Ubuntu
I go to the page and download the file
https://jfrog.com/open-source/#artifactory
The file I download is jfrog-artifactory-oss-6.8.4.zip
Unzip the file and place in working directory.
Change the bin/artifactory.default as follow:
> cat bin/artifactory.def ...
Data Solution 2019(5)Zeppelin Authentication
Limit the other endpoint can only be access by the localhost machine.
> docker run -d -p 127.0.0.1:9870:9870 -p 127.0.0.1:9000:9000 -p 8080:8080 -p 127.0.0.1:4040:4040 -v $(shell pwd)/zeppelin/notebook:/tool/zeppelin/notebook -v $(shell pwd)/zeppelin/c ...
Scrapy Tool Greapy and SpiderKeeper
On my Ubuntu Master Virtual Machine
Check PIP Version
> pip --version
pip 18.1 from /home/carl/.pyenv/versions/3.6.0/lib/python3.6/site-packages/pip (python 3.6)
Install the Gerapy
> pip install gerapy
Check Version
> gerapy -version
0.8.5
In the working ...
Data Solution 2019(4)Investigate my Data in Spark
Create a HDFS directory to store my CSV
> hdfs dfs -mkdir hdfs://localhost:9000/meeting
Put the file there
> hdfs dfs -put ./meetings_only.csv hdfs://localhost:9000/meeting/meetings.csv
> hdfs dfs -put ./users_only.csv hdfs://localhost:9000/ ...
SpringBoot2 and Tomcat Connection Timeout
First of all, in my friends’ project, they did not use last spring boot web-flux.
So we have application.properties as follow
server.port=50310
server.address=0.0.0.0
server.connection-timeout=-1
server.tomcat.connection-timeout=-1
-1 means never timeout I g ...
Data Solution 2019(3)Run Zeppelin in Single Docker
Exception when Start HDFS in Docker
ERROR: Attempting to operate on hdfs namenode as root
ERROR: but there is no HDFS_NAMENODE_USER defined. Aborting operation.
Solution:
Add this to ENV solve the problem.
export HDFS_NAMENODE_USER="root"
...
CXF and Custom Fault Error
Recently, my friend is working on CXF working with Spring boot 2.0.
We get an error as follow:
org.apache.cxf.interceptor.Fault: No binding operation info while invoking unknown method with params unknown.
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(Abstr ...
Data Solution 2019(2)Kafka Single on Docker
Download the binary
> wget http://apache.osuosl.org/kafka/2.1.0/kafka_2.11-2.1.0.tgz
Unzip the file and place in the working directory
Start the single Zookeeper
> bin/zookeeper-server-start.sh config/zookeeper.properties
Start the Kafka Server
> ...
Data Solution(1)Prepare ENV to Parse CSV Data on Single Ubuntu
Java Version
> java -version
java version "1.8.0_161"
Java(TM) SE Runtime Environment (build 1.8.0_161-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.161-b12, mixed mode)
Maven Version
> mvn --version
Apache Maven 3.6.0 ...
Java Object to XML in Order with Annotation
Here is the sample Course.java POJO
package com.sillycat.connectors.quickbooks.models;
import javax.xml.bind.annotation.*;
import java.util.Date;
@XmlRootElement(name = "Course")
@XmlAccessorType(XmlAccessType.FIELD)
@XmlType(propOrder = {
&qu ...
JAVA JSON to XML String in Ordering
In my friend’s project, he requires to convert JSON to XML.
He is using an open source project called JSON-java which is using HashMap in side the implementation which does not support ordering.
So the output XML ordering will be not the same as in JSON.
Here is t ...
Java and Lombok and Eclipse
Download the Lombok from here, the version is 1.18.6 or me currently
https://projectlombok.org/download
> wget https://projectlombok.org/downloads/lombok.jar
Go to find my Eclipse Directory /Applications/Eclipse.app/Contents/MacOS
Place the jar package there
> cp / ...
K8S(2)Install 1.9.11 with dashboard 1.6.3 Version on EC2
On a clean machine
> sudo apt-get update && sudo apt-get install -y apt-transport-https
> curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
> echo "deb https://apt.kubernetes.io/ kubernete ...
K8S(1)Set Up a Customer Cluster
Following this document
https://jimmysong.io/kubernetes-handbook/practice/install-kubernetes-on-ubuntu-server-16.04-with-kubeadm.html
I have 2 Ubuntu machines
192.168.56.101 ubuntu-master
192.168.56.103 ubuntu-dev4
Make Sure and Install Docker
> apt-get install doc ...