`
flex_莫冲
  • 浏览: 1090287 次
  • 性别: Icon_minigender_1
  • 来自: 广州
社区版块
存档分类
最新评论

prediction.io的安裝過程

predicion.io推荐系统大数据 
阅读更多
今天終於將prediction.io裝上VM了。遇到好多問題。

參考：
https://docs.prediction.io/install/
https://docs.prediction.io/install/install-linux/（ubuntu獨立安裝看這裡，推薦）
https://docs.prediction.io/start/download/


如果DB用的不是hbase。如mysql，就不需要安裝Elasticsearch

pio/conf/pio-env.sh 要將DATABASE的設置改成mysql。
最後用pio status 檢查是否安裝成功。

pio命令要執行
export PATH=/home/mochong/PredictionIO-0.9.3/bin:$PATH
才能生效。或者將它放入系統的bin中。
參考：
http://stackoverflow.com/questions/30417280/predictionio-pio-command-not-found-after-install

本機訪問VMWARE的service，必須將vmware的鏈接設置為橋接再重啟vm。
每次重啟要重新獲取vm的ip.
執行 sudo ifconfig 查看。

pio path也要重新設置到PATH中。

查看端口佔用情況：
sudo netstat -antup

demo：
http://192.168.3.111:7070/events.json?accessKey=qt61uzFbC8PZiVPtpTxSEM0nAcIns2tLOFbDiblL2GkkuKvoKokJGMq8IZ1h11FY
剛開始沒消息，返回
{"message":"Not Found"}

當執行rate_item.php後
rate_item.php 
<?php
	require_once("vendor/autoload.php");
	use predictionio\EventClient;
	// 第三個參數是連接超時時間，如果連接超時，屏幕不會打印任何東西出來。
	$client = new EventClient("qt61uzFbC8PZiVPtpTxSEM0nAcIns2tLOFbDiblL2GkkuKvoKokJGMq8IZ1h11FY",
	"http://192.168.3.111:7070", 0, 1); // please update; <URL OF EVENTSERVER> = "http://localhost:7070" by default

	// A user rates an item
	$result = $client->createEvent(array(
	   'event' => 'rate',
	   'entityType' => 'user',
	   'entityId' => 'davidhuang', // please update
	   'targetEntityType' => 'item', 
	   'targetEntityId' => 'sugar_1', // please update
	   'properties' => array('rating'=> "5") // please update
	));
	
	var_dump($result);
?>


執行後，再刷看event.json?xxx 的瀏覽器，顯示新插入的數據內容： 
[{"eventId":"5065b46a5c68428fb8bb0509caa12d93","event":"rate","entityType":"user","entityId":"davidhuang","targetEntityType":"item","targetEntityId":"sugar_1","properties":{"rating":"5"},"eventTime":"2015-07-31T07:27:33.000Z","creationTime":"2015-07-31T07:27:33.000Z"},{"eventId":"e00eb663c6f7417191c7984f31eacb76","event":"rate","entityType":"user","entityId":"davidhuang","targetEntityType":"item","targetEntityId":"sugar_1","properties":{"rating":"5"},"eventTime":"2015-07-31T07:27:44.000Z","creationTime":"2015-07-31T07:27:44.000Z"}]



------------------
PHP Client Setting
1 首先要安裝composer
參考：
http://taojinqu.blog.51cto.com/7849570/1351231
http://my.oschina.net/u/948242/blog/148269

安裝後提示：
Composer successfully installed to: /home/mochong/composer.phar
Use it: php composer.phar


----------------------DATABASE-----------
PIO的表有以下幾個

| pio_event_1                  |
| pio_meta_accesskeys          |
| pio_meta_apps                |
| pio_meta_channels            |
| pio_meta_engineinstances     |
| pio_meta_enginemanifests     |
| pio_meta_evaluationinstances |
| pio_model_models  

pio_event_1:保存event下的entity。
pio_meta_accesskeys:accesskey表
pio_meta_apps:app list 表


ubuntu mysql phpmyadmin安裝參考
http://blog.csdn.net/tecn14/article/details/27515241

使用engine-template
1 下載engin template
pio template get <template-repo-path> <your-new-engine-directory>
參考https://docs.prediction.io/start/download/
如果出现.temlates-cache cannot be written to
则将当前目录的.temlates-cache的文件权限设置为777
sudo chmod 777 .temlates-cache


2 cd 進入下載的engine template 目錄，修改engine.json，將datasource的appname改成自己的appname。 
"datasource": {
    "params" : {
      "name": "sample-handmade-data.txt",
      "appName": "firstpio",
      "eventNames": ["purchase", "view"]
    }
  },


3 安裝python-pip
sudo apt-get install python-pip

4 在engine template 目錄下執行
a python data/import_eventserver.py --access_key NXDJ8UDK73uaynVwbih2MSGNoy07CVlRJXu16tsj5sFMOwHldpjtZ1sbMGougSvB
生成data。

b pio build --verbose
出現錯誤：
Could not retrieve sbt 0.13.7
pio build Return code of previous step is 1. Aborting.
參考：
https://groups.google.com/forum/#!topic/predictionio-user/fllCh8n-0d4
從這裡
https://d29vzk4ow07wi7.cloudfront.net/b407b2a76ad72165f806ac7e7ea09132b951ef53?response-content-disposition=attachment%3Bfilename%3D%22sbt-launch.jar%22&Policy=eyJTdGF0ZW1lbnQiOiBbeyJSZXNvdXJjZSI6Imh0dHAqOi8vZDI5dnprNG93MDd3aTcuY2xvdWRmcm9udC5uZXQvYjQwN2IyYTc2YWQ3MjE2NWY4MDZhYzdlN2VhMDkxMzJiOTUxZWY1Mz9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPWF0dGFjaG1lbnQlM0JmaWxlbmFtZSUzRCUyMnNidC1sYXVuY2guamFyJTIyIiwiQ29uZGl0aW9uIjp7IkRhdGVMZXNzVGhhbiI6eyJBV1M6RXBvY2hUaW1lIjoxNDM4MzMyNTA0fSwiSXBBZGRyZXNzIjp7IkFXUzpTb3VyY2VJcCI6IjAuMC4wLjAvMCJ9fX1dfQ__&Signature=qYv5RDh0C1sHIl7qYfRqyCtugYW1IEJwpO~p6BJjVRqNkGQplkwmpcH6T-1g69YOuI7wsw8xa5Gy3jstjTYjrackGIIdvnfqB3NYQ5yaehuwyexE1grUuhyF2HNXBsXNPuPvPK0X73tHdusEpeIj5FS-nK6u5kRcxaOBIcNO0tYD4db2p~Tcg4a78LzfvFm~r5doZ8JO2Qv9IgidcpdWCB0rDTAYfm2wO-my~bA8Qjh8CFJl43JGz-FCVhYu7wJri1wIulGU~2ZNkZDrVooiEdO4W~luiw5odmme~vXeiV1bFt-xIkPNMa6uK0PVn5ePOucnVeFSS62kyAPdYgvCQw__&Key-Pair-Id=APKAIFKFWOMXM2UMTSFA

下載一個新的sbt-launch.jar替換sbt/sbt-launch-0.13.7.jar

執行後的提示信息： 
[INFO] [Console$] Using existing engine manifest JSON at /home/mochong/PredictionIO-0.9.3/templates/tapster-episode-similar/manifest.json
[INFO] [Console$] Using command '/home/mochong/PredictionIO-0.9.3/sbt/sbt' at the current working directory to build.
[INFO] [Console$] If the path above is incorrect, this process will fail.
[INFO] [Console$] Uber JAR disabled. Making sure lib/pio-assembly-0.9.3.jar is absent.
[INFO] [Console$] Going to run: /home/mochong/PredictionIO-0.9.3/sbt/sbt  package assemblyPackageDependency
[INFO] [Console$] OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory(0x00000000c5550000, 715849728, 0) failed; error='Cannot allocate memory' (errno=12)
[INFO] [Console$] #
[INFO] [Console$] # There is insufficient memory for the Java Runtime Environment to continue.
[INFO] [Console$] # Native memory allocation (malloc) failed to allocate 715849728 bytes for committing reserved memory.
[INFO] [Console$] # An error report file with more information is saved as:
[INFO] [Console$] # /home/mochong/PredictionIO-0.9.3/templates/tapster-episode-similar/hs_err_pid8872.log
[ERROR] [Console$] Return code of previous step is 1. Aborting.


JAVA執行內存不夠。將VM的虛擬內存設置為2048試試。

還要將tempalte中的pio.log,pio.sbt的權限設置為777.不然會提示權限不足。


日誌會保存到pio.log

最終還是失敗了。提示 Could not retrieve sbt 0.13.7


c pio train

d pio deploy

參考：https://docs.prediction.io/deploy/

engin的默認端口是8000

----------------------------------------
export app data
pio export --appid 1 --output firstapp
会在根目录下创建firstapp，数据导出在这里。

-----------------------------------------------------------------
事件模型 event models
All entityType names start with "$" and "pio_" are reserved and shouldn't be used.

All event names start with "$" and "pio_" are reserved
and shouldn't be used as your custom event name (eg. "$set").

所有properties 以$和pio开头的属性，都是pio保留的名称，不能使用

The following special events are reserved for updating entities and their properties:
Special events $set , $unset and $delete are introduced.
"$set" event: Set properties of an entity (also implicitly create the entity). To change properties of entity, you simply set the corresponding properties with value again. The $set events should be created only when:
The entity is first created (or re-create after $delete event), or
Set the entity's existing or new properties to new values (For example, user updates his email, user adds a phone number, item has a updated categories)
"$unset" event: Unset properties of an entity. It means treating the specified properties as not existing anymore. Note that the field properties cannot be empty for $unset event.
"$delete" event: delete the entity.
There is no targetEntityId for these special events.

$set , $unset and $delete 是系统保留的属性。不能使用。

参考：
https://docs.prediction.io/datacollection/eventmodel/

---------------------------------------------------

批量导入数据
PHP SDK还不支持。但是可以将数据导出成json文件。在shell中通过pio import 导入 
 pio import --appid 123 --input my_events.json

导入成功后，就会在对应appid的event表中看到导入的数据
参考：
https://docs.prediction.io/datacollection/batchimport/
-----------------------------------------------

分析工具Ipython

下载安装ipython
http://ipython.org/install.html
执行 
pip install ipython


参考：
http://ipython.org/install.html

http://mindonmind.github.io/2013/02/08/ipython-notebook-interactive-computing-new-era/

------------------------------------------
Event API
1 通过eventID 获取event数据
http://192.168.3.111:7070/events/0d1eb5c7d2eb4f2db85f66c514a46143.json?accessKey=qt61uzFbC8PZiVPtpTxSEM0nAcIns2tLOFbDiblL2GkkuKvoKokJGMq8IZ1h11FY

返回 
{"eventId":"0d1eb5c7d2eb4f2db85f66c514a46143","event":"buy","entityType":"user","entityId":"3","targetEntityType":"item","targetEntityId":"0","properties":{},"eventTime":"2014-11-21T01:04:14.000Z","creationTime":"2015-07-31T12:27:45.000Z"}


2 删除event
curl -i -X DELETE http://localhost:7070/events/<your_eventId>.json?accessKey=<your_accessKey>

3 app的所有event
http://192.168.3.111:7070/events.json?accessKey=qt61uzFbC8PZiVPtpTxSEM0nAcIns2tLOFbDiblL2GkkuKvoKokJGMq8IZ1h11FY

By default, it returns at most 20 events. Use the limit parameter to specify how many events returned (see below). Use cautiously!

In addition, the following optional parameters are supported:

startTime: time in ISO8601 format. Return events with eventTime >= startTime.
untilTime: time in ISO8601 format. Return events with eventTime < untilTime.
entityType: String. The entityType. Return events for this entityType only.
entityId: String. The entityId. Return events for this entityId only.
limit: Integer. The number of record events returned. Default is 20. -1 to get all.
reversed: Boolean. Must be used with both entityType and entityId specified, returns events in reversed chronological order. Default is false.

参考:
https://docs.prediction.io/datacollection/eventapi/#using-event-api

--------------------------------------------------------------------------
##PredictionIO Step by Step Setup Guide v1.0

**A. SERVER SIDE**

1. Ensure you have an appropriate Java version (JDK 7) installed *(if you use quick install command in https://docs.prediction.io/install/, skip steps 1 to 4 and goto step 5)*

java -version

2. Download JDK 7 in http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html and install it (optional)
3. Download PredictionIO

wget https://d8k1yxp8elc6b.cloudfront.net/PredictionIO-0.9.3.tar.gz
tar zxvf PredictionIO-0.9.3.tar.gz
4. Install dependencies

mkdir PredictionIO-0.9.3/vendors
wget http://d3kbcqa49mib13.cloudfront.net/spark-1.3.1-bin-hadoop2.6.tgz
tar zxvfC spark-1.3.1-bin-hadoop2.6.tgz PredictionIO-0.9.3/vendors
5. Create database "pio" *(assume you have database server up and running)*

6. Update pio-env.sh *(especially database settings)*

vi PredictionIO-0.9.3/conf/pio-env.sh
7. Start PredictionIO and dependent services

PredictionIO-0.9.3/bin/pio eventserver &

print "EventServer is ready" if ok.
8. Check PredictionIO status

PredictionIO-0.9.3/bin/pio status
print "your system is all already to go"
9. Download a Template (from http://templates.prediction.io/) i.e. Recommendation template

PredictionIO-0.9.3/bin/pio template get PredictionIO/template-scala-parallel-recommendation <YourNewEngineDir> // i.e. MyRecommendation
cd MyRecommendation
10. Generate an App ID and Access Key

../PredictionIO-0.9.3/bin/pio app new MyApp1
11. List App

../PredictionIO-0.9.3/bin/pio app list

**B. CLIENT SIDE**

*Assume:*
*[weburl] = http://localhost:8888/predictionio-myapp1*
*[webdir] = /Application/MAMP/htdocs/predictionio-myapp1*

1. Download PredictionIO PHP SDK *(By Composer)*

cd [webdir]/
1.1 Create [webdir]/composer.json

{
    "require": {
        "predictionio/predictionio": "~0.8.2"
    }
}
1.2 Install Composer

curl -sS https://getcomposer.org/installer | php -d detect_unicode=Off
1.3 Use Composer to install your dependencies

php composer.phar install
2. Create and update [webdir]/rate-item.php

<?php
require_once("vendor/autoload.php");
use predictionio\EventClient;

$client = new EventClient(<ACCESS KEY>, <URL OF EVENTSERVER>); // please update; <URL OF EVENTSERVER> = "http://localhost:7070" by default

// A user rates an item
$client->createEvent(array(
   'event' => 'rate',
   'entityType' => 'user',
   'entityId' => <USER ID>, // please update
   'targetEntityType' => 'item',
   'targetEntityId' => <ITEM ID>, // please update
   'properties' => array('rating'=> <RATING>) // please update
));
?>
3. Browse http://localhost:8888/predictionio-myapp1/rate-item.php to insert data into database

**C. SERVER SIDE**

12. Check if query event server works fine
Browse http://localhost:7070/events.json?accessKey=[YOUR_ACCESS_KEY]

13. Deploy the engine as a service

cd MyRecommendation

14. Update "appName" in engine.json

...
  "datasource": {
    "params" : {
      "appName": "MyApp1" // must match your app name
    }
  },
  ...

15. Build the engine

../PredictionIO-0.9.3/bin/pio build --verbose

16. Train the predictive model

../PredictionIO-0.9.3/bin/pio train

*Just in case you got java.net.UnknownHostException, please check /etc/hosts. (Read more: http://stackoverflow.com/questions/19330334/hadoop-on-mac-pseudo-node-nodename-nor-servname-provided-or-not-known)*

17. Deploy the engine

../PredictionIO-0.9.3/bin/pio deploy

18. Check the engine status by browsing http://localhost:8000


**D. CLIENT SIDE**

4. Create [webdir]/send-query.php

<?php
require_once("vendor/autoload.php");
use predictionio\EngineClient;

$client = new EngineClient('http://localhost:8000');

$response = $client->sendQuery(array('user'=> 1, 'num'=> 4));
print_r($response);

?>

5. Browse http://localhost:8888/predictionio-myapp1/send-query.php

**E. REFERENCES**

1. PredictionIO - A Quick Intro
https://docs.prediction.io/start/

2. Installing PredictionIO on Linux / Mac OS X
https://docs.prediction.io/install/install-linux/

3. Java SE Development Kit 7 Downloads
http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html

4. PredictionIO Engine Templates
http://templates.prediction.io/

5. Quick Start - Recommendation Engine Template
https://docs.prediction.io/templates/recommendation/quickstart/

6. PredictionIO-PHP-SDK
https://github.com/PredictionIO/PredictionIO-PHP-SDK
  • 大小: 19.1 KB
  • 大小: 8.3 KB
  • 大小: 18 KB
分享到：
| ubuntu安裝redis
评论
发表评论

您还没有登录,请您登录后再发表评论

相关推荐

    prediction io

    7. **部署与管理**：Prediction IO 提供了安装脚本和配置文件，便于在不同环境（如本地、云服务器）上部署和管理。此外，它还支持集群部署，以应对大规模数据和高并发场景。 8. **版本控制与持续集成**：作为一个...

    注塑Craft.io启动过程的质量分析与预测

    9. 相关关键词：研究涉及到的关键词包括多阶段批量过程（Multiphase batch processes）、启动过程（Start-up process）、质量预测（Quality prediction）和注塑成型过程（Injection molding process）。 在撰写相关...

    petolau.github.io：有关R中时间序列数据挖掘的博客

    `stl()`函数用于执行这个过程，帮助识别各部分特征。 5. **时间序列建模**：ARIMA（自回归整合滑动平均模型）是时间序列预测的经典模型，`auto.arima()`函数可以自动选择合适的模型参数。还有状态空间模型（如...

    Surprise - Prediction Algorithms（预测方法）

    Surprise库是一个用Python编写的轻量级推荐系统库，旨在简化推荐系统的开发过程，并提供多种经典的预测算法。接下来，我们将详细讨论每种算法的功能、工作原理以及如何使用它们来构建高效的推荐系统。 ### 基础算法...

    predictionIOEventServer.zip

    集成Hbase到Prediction IO的过程中，SpringBoot框架的使用起到了关键作用。SpringBoot简化了Java应用的开发，提供了一种快速构建独立应用的方式，同时支持自动配置和容器化部署。通过SpringBoot，我们可以方便地在...

    Prediction

    这可能包括从CSV、Excel文件或数据库中读取数据，使用C#的库如`System.IO`和`CsvHelper`。数据预处理步骤包括清洗（去除缺失值和异常值）、归一化、编码类别变量等。 2. **线性回归**：C#可以用于构建简单的线性...

    Prediction of household and commercial BMW

    这个过程可能涉及到对历史废物生成数据的分析以及对当前文献的回顾，以确定不同类型家庭和商业设施典型的废物生成水平。 研究结果结合了来自家庭和商业设施的预测，从而给出了整个都柏林地区的BMW总体估计。这些...

    BERT学习1

    BERT首先在大规模无标注文本数据上进行预训练，通过两个任务：掩码语言模型（Masked Language Model, MLM）和下一句预测（Next Sentence Prediction, NSP），学习通用的语言表示。然后，在特定任务如问答、情感分析...

    hadoop学习-基于hive的航空公司客户价值的LRFCM模型案例数据源

    本案例聚焦于如何使用Hive构建一个LRFCM（Logistic Regression for Churn Prediction and Cluster Mining，逻辑回归流失预测与聚类挖掘）模型，以评估航空公司的客户价值。我们将探讨Hadoop生态系统中的Hive组件，...

    C++eigen库和dlib库使用笔记.md

    安装过程中还需要安装必要的依赖库，如OpenCV等。 ##### 3.3 图像加载与处理 Dlib支持图像的加载、显示和转换等功能，这为后续的图像处理提供了便利。 **示例代码：** ```cpp #include &lt;dlib/image_io.h&gt; #include...

    大型立式淬火炉中铝合金热处理温度控制系统

    为了满足淬火Craft.io的技术要求，已经开发了一种温度控制系统，用于在大型立式淬火炉中对铝合金工件进行热处理。 在本文中，建立了一个空气传热模型以获得温度梯度分布，从而可以根据可用的热电偶温度来预测无法...

    基于matplotlib中ion()和ioff()的使用详解

    此外，还提供了一个示例，演示如何在训练神经网络的过程中，利用交互模式实时更新图形： ```python with tf.Session() as sess: sess.run(init) fig = plt.figure() ax = fig.add_subplot(1,1,1) ax.scatter(x_...

    ppmtool

    根据提供的信息，我们可以推测这可能是一个用于处理图像、性能监控或者项目管理的工具，因为"ppm"在不同的上下文中可以指代多种含义，比如“像素-per-inch”（图像分辨率）或“Performance Prediction Model”（性能...

    Pytorch专题实战——线性回归(Linear Regression)

    print(f'Prediction after training: f(5)={model(X_test).item():.3f}') ``` 通过以上步骤，我们可以用PyTorch实现一个线性回归模型，并训练它来拟合给定的数据。这个模型可以处理更复杂的任务，只要输入和输出...

Magicbox
Global site tag (gtag.js) - Google Analytics