项目典型数据应用缓存的jvm之旅

fruitking

浏览: 282913 次
性别:
来自: 杭州

最近访客更多访客>>

uule

wangyy

catchid2003

u012363178

博主相关

博客

微博

相册

留言

关于我

文章分类

社区版块

存档分类

博客分类：

实际应用

JVM quartz log4j Spring Cache

在很多项目中，有一些数据量不小，但是也不是很大的，且数据更新频率很小的数据
这种情况下我们一般会自己缓存这些数据，现在我们使用spring+quartz的方式来处理，
即以定时更新这部分数据，以达到在某个时段可以重复使用内存缓存数据，不必访问数据库或者其它直接数据源
先定义一个任务类，以执行定时更新操作，这里从数据库查询一个国家，城市数据作为一个类的静态成员数据
这个任务类：AreaCodeJob.java

package com.fruitking.cache.job;

import java.util.List;

import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.quartz.JobExecutionContext;
import org.quartz.JobExecutionException;
import org.springframework.scheduling.quartz.QuartzJobBean;

import com.fruitking.entity.Country;
import com.fruitking.entity.City;
import com.fruitking.service.IAreaCodeService;

public class AreaCodeJob extends QuartzJobBean{
	
	private static final Log log = LogFactory.getLog(AreaCodeJob.class);
	private IAreaCodeService areaCodeService;
	
	public void setAreaCodeService(IAreaCodeService areaCodeService) {
		this.areaCodeService = areaCodeService;
	}
	
	protected void executeInternal(JobExecutionContext arg0)throws JobExecutionException {
		try {
			long changeMemory = Runtime.getRuntime().freeMemory();
			//为了获取所有国家信息
			List<Country> countryCodeList = areaCodeService.findCountrys();
			//为了获取所有城市信息
			List<City> cityCodeList = areaCodeService.findCitys();
			//更新缓存内容
			DBDataCacheContext.setCountryCodeList(countryCodeList);
			DBDataCacheContext.setCityCodeList(cityCodeList);
			if(log.isInfoEnabled()){
				long freeMemory = Runtime.getRuntime().freeMemory();
				long maxMemory = Runtime.getRuntime().maxMemory();
				long totalMemory = Runtime.getRuntime().totalMemory();
				changeMemory = changeMemory - freeMemory;
				String jvmIndf = "start to end...jvm free memory change:"+changeMemory+"B also:"+(changeMemory/1024)+"KB,free memory is:"+freeMemory + ",total memory is:"+totalMemory+",max memory is:"+maxMemory;
				log.info(jvmIndf);
			}
		} catch (Exception e) {
			log.error(e.getMessage(), e);
		}
	}

}

缓存类的编写DBDataCacheContext.java

package com.fruitking.cache.job;

import java.util.List;

import com.fruitking.entity.Country;
import com.fruitking.entity.City;

public class DBDataCacheContext {
	
	private static List<Country> countryCodeList = null;//所有国家信息
	private static List<City> cityCodeList = null;//所有城市信息

	public static List<Country> getCountryCodeList() {
		return countryCodeList;
	}

	public static void setCountryCodeList(
			List<Country> countryCodeList) {
		DBDataCacheContext.countryCodeList = null;
		DBDataCacheContext.countryCodeList = countryCodeList;
	}

	public static List<City> getCityCodeList() {
		return cityCodeList;
	}

	public static void setCityCodeList(
			List<City> cityCodeList) {
		DBDataCacheContext.cityCodeList = null;
		DBDataCacheContext.cityCodeList = cityCodeList;
	}
}

spring的定时配置文件

<?xml version="1.0" encoding="UTF-8"?>
<beans
	xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd">

<!--spring定时触发， Quartz 配置-->   
<!-- 定时更新折扣信息 -->
<bean id="areaCodeJob" class="org.springframework.scheduling.quartz.JobDetailBean">
	<property name="jobClass">
		<value>com.fruitking.cache.job.AreaCodeJob</value>
	</property>
	<property name="jobDataAsMap">
		<map>
			<entry key="areaCodeService"><ref bean="areaCodeService"/></entry>
		</map>
	</property>
</bean>

<bean id="areaCodeJobTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">   
    <property name="jobDetail">   
        <ref bean="areaCodeJob"/>   
    </property>   
    <property name="cronExpression">   
    <!--eg: 每小时整点跟新一次,表达式为"0 0 * * * ?" -->
        <value>0/5 * * * * ?</value>
    </property>   
</bean>
<!-- 触发器调度控制器 -->   
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">   
    <property name="triggers">
        <list>
         <ref bean="areaCodeJobTrigger"/>
        </list>
    </property>   
</bean>
</beans>

这样就可以在其它类里面使用这个国家，城市信息数据了，直接当静态数据使用，这样就大大提高了速度

log4j.properties配置文件设置一下
log4j.rootLogger=error,stdout
log4j.appender.stdout.layout.ConversionPattern=%-d{HH:mm:ss} - %m%n
log4j.logger.com.fruitking.cache.job=info

这样启动相应工程，观察jvm的内存变化
我这里的数据量大约在7000条左右，按照字节算应该在12M左右的数据量，但是通过jvm观察，内存实际一次增加15-20M
我想这可能是程序执行过程的局部变量产生的吧
我这里为了测试每5秒钟更新一次数据，实际使用中可能每半小时，每小时，甚至每天，一周才更新一次数据，可以通过corn表达式配置
运行的jvm内存数据变化如下：
10:20:53 - start to end...jvm free memory change:-8535944B also:-8335KB,free memory is:464688232
10:20:58 - start to end...jvm free memory change:20714696B also:20229KB,free memory is:442974016
10:21:02 - start to end...jvm free memory change:18470480B also:18037KB,free memory is:423619808
10:21:07 - start to end...jvm free memory change:21115288B also:20620KB,free memory is:429755752
10:21:12 - start to end...jvm free memory change:18710776B also:18272KB,free memory is:409949680
10:21:17 - start to end...jvm free memory change:-7640992B also:-7461KB,free memory is:416865832
10:21:22 - start to end...jvm free memory change:20315064B also:19838KB,free memory is:395625680
10:21:27 - start to end...jvm free memory change:19071784B also:18624KB,free memory is:375837648
10:21:32 - start to end...jvm free memory change:-7389080B also:-7215KB,free memory is:382382832
10:21:37 - start to end...jvm free memory change:19783216B also:19319KB,free memory is:361913736
10:21:42 - start to end...jvm free memory change:-5315160B also:-5190KB,free memory is:366734176
10:21:47 - start to end...jvm free memory change:20523832B also:20042KB,free memory is:345657112
10:21:52 - start to end...jvm free memory change:19157344B also:18708KB,free memory is:325855784
10:21:57 - start to end...jvm free memory change:-7493720B also:-7318KB,free memory is:332798168
10:22:02 - start to end...jvm free memory change:20266552B also:19791KB,free memory is:311923072
10:22:07 - start to end...jvm free memory change:19279584B also:18827KB,free memory is:292246632
10:22:12 - start to end...jvm free memory change:-7546800B also:-7369KB,free memory is:299198448
10:22:17 - start to end...jvm free memory change:19976304B also:19508KB,free memory is:278662488
10:22:22 - start to end...jvm free memory change:-7270136B also:-7099KB,free memory is:285511848
10:22:27 - start to end...jvm free memory change:20545944B also:20064KB,free memory is:264410968
10:22:32 - start to end...jvm free memory change:19383184B also:18928KB,free memory is:244694904
10:22:37 - start to end...jvm free memory change:-7212344B also:-7043KB,free memory is:251415320
10:22:42 - start to end...jvm free memory change:19974280B also:19506KB,free memory is:230890280
10:22:47 - start to end...jvm free memory change:18752272B also:18312KB,free memory is:209553416
10:22:52 - start to end...jvm free memory change:-7150144B also:-6982KB,free memory is:216190760
10:22:57 - start to end...jvm free memory change:19399488B also:18944KB,free memory is:196329776
10:23:02 - start to end...jvm free memory change:-7469600B also:-7294KB,free memory is:203393160
10:23:07 - start to end...jvm free memory change:20435936B also:19956KB,free memory is:182441944
10:23:12 - start to end...jvm free memory change:19334896B also:18881KB,free memory is:162776104
10:23:17 - start to end...jvm free memory change:-7098288B also:-6931KB,free memory is:169379904
10:23:22 - start to end...jvm free memory change:20187808B also:19714KB,free memory is:148749040
10:23:27 - start to end...jvm free memory change:19275120B also:18823KB,free memory is:129105136
10:23:32 - start to end...jvm free memory change:-6974952B also:-6811KB,free memory is:135752520
10:23:37 - start to end...jvm free memory change:19390936B also:18936KB,free memory is:115898104
10:23:42 - start to end...jvm free memory change:-7392224B also:-7218KB,free memory is:122888592
10:23:47 - start to end...jvm free memory change:20411232B also:19932KB,free memory is:101997272
10:23:52 - start to end...jvm free memory change:19200584B also:18750KB,free memory is:82457368
10:23:57 - start to end...jvm free memory change:-6989048B also:-6825KB,free memory is:88964600
10:24:02 - start to end...jvm free memory change:20093744B also:19622KB,free memory is:68436232
10:24:07 - start to end...jvm free memory change:19069808B also:18622KB,free memory is:48973840
10:24:12 - start to end...jvm free memory change:-6860432B also:-6699KB,free memory is:55555312
10:24:17 - start to end...jvm free memory change:19083904B also:18636KB,free memory is:35998224
10:24:22 - start to end...jvm free memory change:-7115224B also:-6948KB,free memory is:42700944
10:24:27 - start to end...jvm free memory change:20389944B also:19912KB,free memory is:21828952,total memory is:532807680,max memory is:1065484288
10:24:33 - start to end...jvm free memory change:-396309648B also:-387021KB,free memory is:417800024,total memory is:532807680,max memory is:1065484288
10:24:37 - start to end...jvm free memory change:-3880768B also:-3789KB,free memory is:421199168,total memory is:532807680,max memory is:1065484288
10:24:42 - start to end...jvm free memory change:20413072B also:19934KB,free memory is:400347744
10:24:47 - start to end...jvm free memory change:-5273152B also:-5149KB,free memory is:403160512
10:24:52 - start to end...jvm free memory change:20586320B also:20103KB,free memory is:382110152
10:24:57 - start to end...jvm free memory change:19145032B also:18696KB,free memory is:362503656

这里从中可以总结几点：
1。随着数据的加载，jvm内存使用量不断增加，每次近20M（因为这个数据很大），这个比我想的要夸张点
2。jvm内存的回收绝大部分实在内存可使用量很低的时候才大量回收垃圾数据，其它时候虽然偶尔也回收，回收幅度很小
3。总的内存数量和最大内存数量值没有变化