- 浏览: 2611573 次
- 性别:
- 来自: 广州
文章分类
- 全部博客 (880)
- 每日总结 (26)
- java (37)
- lucene 2.4源码学习 (11)
- 庖丁分词的源码分析 (5)
- cindy源码阅读 (10)
- jetty (3)
- java基础 (49)
- hadoop (37)
- python (7)
- 那些蛋疼的事 (15)
- 测试 (5)
- spring (3)
- redis (4)
- git (4)
- 我的开源项目 (2)
- linux (15)
- mysql (8)
- bootsharp (1)
- idea (1)
- Disruptor源码分析 (11)
- 高并发 (22)
- jdk (4)
- 领域驱动 (1)
- 坑 (6)
- 表达式框架 (7)
- 游戏 (3)
- Guava (1)
- 缓存 (1)
- 数据库 (1)
- 游戏项目 (3)
- java8 (1)
最新评论
-
hsluoyz:
PyCasbin是一个用Python语言打造的轻量级开源访问控 ...
权限管理的一个简单后台 -
liuyachao111:
谢谢 问题解决了
对实体 "characterEncoding" 的引用必须以 ';' 分隔符结尾 -
jnjeC:
多谢博主分享,在配置文件里的&也要改成& ...
对实体 "characterEncoding" 的引用必须以 ';' 分隔符结尾 -
大维啊:
同志,你这不行啊!
java 的 AccessController.doPrivileged使用 -
lisha2009:
解决了我的问题,多谢博主分享!
对实体 "characterEncoding" 的引用必须以 ';' 分隔符结尾
原文:http://java-performance.info/possible-memory-leak-in-multimap/
典型的不作就不会死的节奏。这个最大的误区就是在get的方法中加入了put的操作。责任不清楚,就容易导致内存泄露。
下面是原文:
A possible memory leak in the manual MultiMap implementation
by Mikhail Vorontsov
Pure Java 6 and 7
In this short article I will describe a junior level memory leak I have recently seen in a pure JDK application (no 3rd party libs were allowed).
Suppose you have a map from String identifiers to some Collections, for example a set of String properties of such identifiers: Map<String, Set<String>>. The actual type of the inner collection does not matter for this article – it should just be a Collection. Such collections are generally called multimaps.
The following method was initially written to obtain the inner set by its identifier:
1
2
3
4
5
6
7
8
9
10
11
12
private final Map<String, Set<String>> m_ids = new HashMap<String, Set<String>>( 4 );
private Set<String> getSetByNameFaulty( final String id )
{
Set<String> res = m_ids.get( id );
if ( res == null )
{
res = new HashSet<String>( 1 );
m_ids.put( id, res );
}
return res;
}
This method checks if an identifier is already present in the map and either returns the corresponding Set or allocates a new one and adds it into the map. This method is useful for populating our map:
1
2
3
4
5
6
7
private void populateJava67()
{
getSetByNameFaulty( "id1" ).add( "property1" );
getSetByNameFaulty( "id1" ).add( "property2" );
getSetByNameFaulty( "id1" ).add( "property3" );
getSetByNameFaulty( "id2" ).add( "property1" );
}
The next step while writing a program would be to add some accessors to our map, like the following one:
1
2
3
4
private boolean hasPropertyFaulty( final String id, final String property )
{
return getSetByNameFaulty( id ).contains( property );
}
This method looks good and is unlikely to be caught by any code quality tools. Unfortunately, it has a major flaw: if you will query properties of unknown identifier, a new empty set will be allocated in our map inside getSetByNameFaulty method. This is definitely a not wanted side effect. Instead we should let our new getSetByName method know if we want to write something to the returned set:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
private Set<String> getSetByName( final String id, final boolean isWrite )
{
Set<String> res = m_ids.get( id );
if ( res == null )
{
if ( isWrite )
{
res = new HashSet<String>( 1 );
m_ids.put( id, res );
}
else
res = Collections.emptySet();
}
return res;
}
private boolean hasProperty( final String id, final String property )
{
return getSetByName( id, false ).contains( property );
}
private void populateJava67Better()
{
getSetByName( "id1", true ).add( "property1" );
getSetByName( "id1", true ).add( "property2" );
getSetByName( "id1", true ).add( "property3" );
getSetByName( "id2", true ).add( "property1" );
}
Java 8
Java 8 has added the following method to the Map interface (actually, many more methods were added, but we are interested in this one only):
1
V getOrDefault(Object key, V defaultValue)
This method returns either a value from the map or a default value if a key is not present in the map. The map itself is not updated on this call. It will allow us to simplify the read accesses to the map to
1
2
3
4
private boolean hasPropertyJava8( final String id, final String property )
{
return m_ids.getOrDefault( id, Collections.<String>emptySet() ).contains( property );
}
Google Guava
Google Guava library defines a MultiMap interface and provides its several implementations (number of those is actually similar to a number of Map implementations in JDK collections framework).
MultiMap.get defined as
1
Collection<V> get(K key)
It returns all values associated with a given key (or an empty collection if nothing is associated). Here is a possible client code implementation using Google Guava. As you can see, we don’t have to explicitly care about inner collection:
1
2
3
4
5
6
7
8
9
10
11
private final Multimap<String, String> m_idsMM = HashMultimap.create();
private void addPropertyGuava( final String id, final String property )
{
m_idsMM.put( id, property );
}
private boolean hasPropertyGuava( final String id, final String property )
{
return m_idsMM.get( id ).contains( property );
}
Scala 2.10
Scala is one of most popular JVM languages. Its current release, 2.10, provides user a rich library of collections.
The following example mixes in Scala MultiMap trait with an ordinary Scala HashMap. It uses MapLike.getOrElseUpdate method for initialization of absent key-value pairs.
object MultimapTest {
val m_ids = new mutable.HashMap[String, mutable.Set[String]] with mutable.MultiMap[String, String]
def addProperty( id:String, property:String) {
m_ids.getOrElseUpdate(id, new mutable.HashSet[String]())+=property
}
def hasProperty( id:String, property:String) = m_ids(id).contains(property)
def main( args:Array[String])
{
addProperty("id1", "prop1")
addProperty("id1", "prop2")
addProperty("id1", "prop3")
addProperty("id2", "prop1")
println(hasProperty("id1", "prop2"))
println(hasProperty("id1", "prop4"))
println(hasProperty("id2", "prop2"))
println(hasProperty("id2", "prop4"))
}
}
This code will print true followed by 3 false.
Summary
As you have seen, it is quite easy to miss a memory leak while implementing a multilevel map. You need to be careful and split read and write accesses to the outer map.
Newer frameworks and languages, like Google Guava, Java 8 and Scala already provide you more convenient syntax and wider choice of collections thus allowing you to avoid possible memory leaks in the multilevel maps.
典型的不作就不会死的节奏。这个最大的误区就是在get的方法中加入了put的操作。责任不清楚,就容易导致内存泄露。
下面是原文:
A possible memory leak in the manual MultiMap implementation
by Mikhail Vorontsov
Pure Java 6 and 7
In this short article I will describe a junior level memory leak I have recently seen in a pure JDK application (no 3rd party libs were allowed).
Suppose you have a map from String identifiers to some Collections, for example a set of String properties of such identifiers: Map<String, Set<String>>. The actual type of the inner collection does not matter for this article – it should just be a Collection. Such collections are generally called multimaps.
The following method was initially written to obtain the inner set by its identifier:
1
2
3
4
5
6
7
8
9
10
11
12
private final Map<String, Set<String>> m_ids = new HashMap<String, Set<String>>( 4 );
private Set<String> getSetByNameFaulty( final String id )
{
Set<String> res = m_ids.get( id );
if ( res == null )
{
res = new HashSet<String>( 1 );
m_ids.put( id, res );
}
return res;
}
This method checks if an identifier is already present in the map and either returns the corresponding Set or allocates a new one and adds it into the map. This method is useful for populating our map:
1
2
3
4
5
6
7
private void populateJava67()
{
getSetByNameFaulty( "id1" ).add( "property1" );
getSetByNameFaulty( "id1" ).add( "property2" );
getSetByNameFaulty( "id1" ).add( "property3" );
getSetByNameFaulty( "id2" ).add( "property1" );
}
The next step while writing a program would be to add some accessors to our map, like the following one:
1
2
3
4
private boolean hasPropertyFaulty( final String id, final String property )
{
return getSetByNameFaulty( id ).contains( property );
}
This method looks good and is unlikely to be caught by any code quality tools. Unfortunately, it has a major flaw: if you will query properties of unknown identifier, a new empty set will be allocated in our map inside getSetByNameFaulty method. This is definitely a not wanted side effect. Instead we should let our new getSetByName method know if we want to write something to the returned set:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
private Set<String> getSetByName( final String id, final boolean isWrite )
{
Set<String> res = m_ids.get( id );
if ( res == null )
{
if ( isWrite )
{
res = new HashSet<String>( 1 );
m_ids.put( id, res );
}
else
res = Collections.emptySet();
}
return res;
}
private boolean hasProperty( final String id, final String property )
{
return getSetByName( id, false ).contains( property );
}
private void populateJava67Better()
{
getSetByName( "id1", true ).add( "property1" );
getSetByName( "id1", true ).add( "property2" );
getSetByName( "id1", true ).add( "property3" );
getSetByName( "id2", true ).add( "property1" );
}
Java 8
Java 8 has added the following method to the Map interface (actually, many more methods were added, but we are interested in this one only):
1
V getOrDefault(Object key, V defaultValue)
This method returns either a value from the map or a default value if a key is not present in the map. The map itself is not updated on this call. It will allow us to simplify the read accesses to the map to
1
2
3
4
private boolean hasPropertyJava8( final String id, final String property )
{
return m_ids.getOrDefault( id, Collections.<String>emptySet() ).contains( property );
}
Google Guava
Google Guava library defines a MultiMap interface and provides its several implementations (number of those is actually similar to a number of Map implementations in JDK collections framework).
MultiMap.get defined as
1
Collection<V> get(K key)
It returns all values associated with a given key (or an empty collection if nothing is associated). Here is a possible client code implementation using Google Guava. As you can see, we don’t have to explicitly care about inner collection:
1
2
3
4
5
6
7
8
9
10
11
private final Multimap<String, String> m_idsMM = HashMultimap.create();
private void addPropertyGuava( final String id, final String property )
{
m_idsMM.put( id, property );
}
private boolean hasPropertyGuava( final String id, final String property )
{
return m_idsMM.get( id ).contains( property );
}
Scala 2.10
Scala is one of most popular JVM languages. Its current release, 2.10, provides user a rich library of collections.
The following example mixes in Scala MultiMap trait with an ordinary Scala HashMap. It uses MapLike.getOrElseUpdate method for initialization of absent key-value pairs.
object MultimapTest {
val m_ids = new mutable.HashMap[String, mutable.Set[String]] with mutable.MultiMap[String, String]
def addProperty( id:String, property:String) {
m_ids.getOrElseUpdate(id, new mutable.HashSet[String]())+=property
}
def hasProperty( id:String, property:String) = m_ids(id).contains(property)
def main( args:Array[String])
{
addProperty("id1", "prop1")
addProperty("id1", "prop2")
addProperty("id1", "prop3")
addProperty("id2", "prop1")
println(hasProperty("id1", "prop2"))
println(hasProperty("id1", "prop4"))
println(hasProperty("id2", "prop2"))
println(hasProperty("id2", "prop4"))
}
}
This code will print true followed by 3 false.
Summary
As you have seen, it is quite easy to miss a memory leak while implementing a multilevel map. You need to be careful and split read and write accesses to the outer map.
Newer frameworks and languages, like Google Guava, Java 8 and Scala already provide you more convenient syntax and wider choice of collections thus allowing you to avoid possible memory leaks in the multilevel maps.
发表评论
-
获取字符长度的正确姿势
2017-05-23 16:09 1133public static void main(String[ ... -
解决tomcat中反序列化找不到class
2017-05-19 09:59 2180tomcat反序列化的过程中一直报ClassNotFoundE ... -
java的sun.jnu.encoding有什么用
2017-02-10 15:45 5529目前看到的影响有两个:影响类名的读取和Main方法参数的读取。 ... -
jsckson序列化处理泛型
2017-01-10 15:02 3393我有这样两个类 package com.vipshop. ... -
java的double乘法精度问题
2016-12-22 09:31 5603项目中实际的代码,我们实际的金额单位是元,精确到分,另外一个系 ... -
Calendar.getInstance()的坑
2016-12-06 16:50 6003Calendar.getInstance()看起来应该是个单例 ... -
针对jquery的when方法做的应变
2016-10-13 17:09 1111需求:a,b两个任务都处理(不管a,b是成功还是失败)后,执行 ... -
http的501错误
2016-10-09 15:37 8738普通的url请求是get put之类的,如果是乱七八糟的,比如 ... -
java对象初始化的顺序
2016-10-08 17:18 1037public class Son extends F ... -
java序列化框架性能比较
2016-05-24 09:22 32911. Kryo 2. FST 3. java 原生序列化 ... -
java 关闭main方法中的定时器线程(2)
2016-05-20 15:49 1820import java.util.concurrent ... -
java 关闭main方法中的定时器线程
2016-05-20 15:29 1232public class TestTreadClose { ... -
java用cyclicBarrier来实现Phaser的分段功能
2015-01-26 10:22 1602cyclicBarrier是有自动重置功能的,我们可以用这个功 ... -
Java的Integer是由缓存决定的,我们可以改变缓存来改变Integer
2015-01-25 16:00 1671import java.lang.reflect.Fi ... -
BigDecimal做四舍五入的坑
2015-01-12 10:23 4195BigDecimal decimal = new BigD ... -
对实体 "characterEncoding" 的引用必须以 ';' 分隔符结尾
2015-01-03 11:27 36339在hibernate启动的时候报了个错: 对实体 " ... -
mvel表达式遇到的坑
2014-12-31 18:02 5233简单的说是:mvel中int和一个double做乘法,是可能出 ... -
Java 泛型系统的不协变问题和类型推断
2014-11-11 21:16 1261原文:http://jerrypeng.me/2013/02/ ... -
spring启动的时候尝试多线程发生的死锁
2014-09-12 11:12 3954具体的死锁线程dump: Found one Java-l ... -
java方法的syntethic类型
2014-04-15 19:16 1189我们一般说java的方法有public private,但其实 ...
相关推荐
在C++编程中,`std::multimap`是标准模板库(STL)中的一种关联容器。它是一个可以存储键值对的数据结构,其中键值可以重复。与`std::map`不同,`std::multimap`允许有多个具有相同键的元素。在“好友列表”这个场景中...
在STL容器中,`multimap`是一种关联容器,它允许元素以键值对的形式存储,并且一个键可以对应多个值。与`map`不同的是,`map`保证了键的唯一性,而`multimap`则允许重复的键。 在`multimap`中,每个元素都是一个...
本程序用c++标准库中的快速排序算法以及容器map和multimap实现了对文件中出现频率靠前的单词进行了统计,至于文件大小取决于你机器所剩的内存数,如果文件数据时海量的(上亿)则本程序可能不适合,需要按照海量数据...
通过multimap进行信息的插入 key(部门编号) value(员工) 分部门显示员工信息 实现步骤: 创建10名员工,放到vector中 遍历vector容器,取出每个员工,进行随机分组 分组后,将员工部门编号作为key,具体员工作为...
多图MultiMAP是一种集成单细胞多组学的方法。 MultiMAP也可以用于批量校正。 更多详细信息可在我们的。安装我们正在努力让包装上pip尽快。 同时享受此占位符解决方案! git clone ...
多图这是语言缺少的multimap集合(这也是创建适当的库/包的一种简单做法)。 多重映射(有时也称为多重哈希或多重字典)是映射或关联数组抽象数据类型的概括,其中多个以上值可以与给定键相关联并返回给定键。 此...
- **元素插入**:`std::multimap` 提供了多种插入元素的方法,如 `insert()`、`emplace()` 等。这些方法可以确保即使插入重复键也不会违反容器的排序规则。 - **查找操作**:由于元素按键排序,`std::multimap` 提供...
在C++编程中,`Multimap`是一种关联容器,它提供了键值对的无序存储。`Multimap`是标准模板库(STL)的一部分,由`<map>`头文件定义。与`Map`不同,`Multimap`允许键值对的重复,也就是说,同一个键可以映射到多个值...
multimap.yml
这个问题可以通过使用C++中的`map`和`multimap`数据结构来解决。 首先,我们需要理解`map`和`multimap`的基本概念。`map`是一个关联容器,它包含唯一键值对,这里的键是单词,值是对应的出现次数。`map`内部默认...
总的来说,基于`multimap`的动态内存分配算法是一种创新的方法,它结合了数据结构的优势,提高了内存分配的效率和内存管理的优化程度。这种算法对于编写高效、内存利用率高的软件,尤其是那些内存资源紧张的环境,...
标题中的“multimap实现tlv”指的是在C++编程中使用`std::multimap`容器来实现一种特定的数据结构——TLV(Tag-Length-Value)。TLV是一种常见于网络协议和数据序列化中的数据表示方式,它将数据分为三个部分:标签...
一、multimap(一对多索引) C++ multimap和map所支持的操作相同(除了multimap不支持下标运算),但是multimap允许重复的元素。 完整程序代码: /*请务必运行以下程序后对照阅读*/ ///头文件依旧是map #include #...
同样,`multimap`内部也是按照键的顺序排列,可以使用`lower_bound`和`upper_bound`方法找到特定键的范围,或者使用`equal_range`获取键的所有匹配项。 **红黑树**: 红黑树是一种自平衡二叉查找树,它在保持查找...
在C++标准库中,模板被广泛应用于STL(Standard Template Library,标准模板库),其中包括了各种容器如vector、map、multimap、set和multiset。这些容器都是模板类,它们提供了高效的数据组织和操作方式。 1. **...
c++ vs2019 cpp20规范的STL库的map与multimap源码分析
标题"MXTWebMultipleMaps_carefulkrl_multimap_SharpMapC#_zip_webmap_"揭示了我们将重点讨论的关键技术:C#、SharpMap库以及多图层地图的实现。 SharpMap是一款开源的、基于.NET Framework的GIS库,它允许开发者...
用AVL-tree数据结构作为底层机制,以STL底层空间配置器和iterator_traits编程技法实作出一个独立的关联式容器(map, set, multimap, multiset),并对外提供接口实现和STL完全兼容的容器。