DynamoDB and Query/Scan Limit
Recently, I found one issue in my Project for DynamoDB.
I do query by index and some filters. But even I put
SELECT: “COUNT”
DynamoDB will only return me parts of the whole records. It is said the limit is 4M. Even the count is part of the whole results.
I need to do pagination and add up all the count together to get the final total count. Or use a list to hold the Items and add up all the Items to get the while results.
Here is the example of query with pagination
export const countCurrentMeetingsByGroup = async (groupUUID: string) => {
const tempMeeting = false;
let lastEvaluatedKey = null;
let countTotal = 0;
do {
const params: any = {
TableName: meetingTableName,
IndexName: meetingTableIndex,
KeyConditionExpression: "#groupUUID = :groupUUID",
FilterExpression: "#tempMeeting = :tempMeeting AND #action <> :action ",
ExpressionAttributeNames:{
"#groupUUID": "groupUUID",
"#tempMeeting": "tempMeeting",
"#action": "action"
},
ExpressionAttributeValues: {
":groupUUID": groupUUID,
":tempMeeting": tempMeeting,
":action": "DELETED"
},
Select: 'COUNT'
};
if (lastEvaluatedKey) {
params['ExclusiveStartKey'] = lastEvaluatedKey;
}
console.log('getting countCurrentMeetingsByGroup from DB with params: ', params);
const data = await dynamoDb.query(params).promise();
lastEvaluatedKey = idx(data, _ => _.LastEvaluatedKey);
const count = idx(data, _ => _.Count);
console.log(`data=${JSON.stringify(data)}`);
console.log(`lastEvaluatedKey=${JSON.stringify(lastEvaluatedKey)}`);
console.log(`count=${count}`);
if (count) {
countTotal = countTotal + count;
}
console.log('countCurrentMeetingsByGroup load from DB, meeting count = ' + count + ' countTotal = ' + countTotal);
} while (lastEvaluatedKey);
return countTotal;
};
The most important part is
lastEvaluatedKey = idx(data, _ => _.LastEvaluatedKey);
References:
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-Select
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Limit
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-Select
https://egkatzioura.com/2016/07/20/query-dynamodb-items-with-node-js-part-2/
https://stackoverflow.com/questions/15543418/how-to-get-the-result-of-describetable-amazon-dynamodb-method-in-node-js
https://github.com/awslabs/dynamodb-data-mapper-js/issues/72
分享到:
相关推荐
:onyx/ident :dynamodb/scan :onyx/type :input :onyx/medium :dynamodb-scan :onyx/consumption :concurrent :dynamodb/config client-opts :dynamodb/scan-options { :return :all-attributes } :dynam
Alternator.zip,用于测试的模拟本地dynamodb实例一个本地运行用于测试的模拟dynamodb-已弃用,请使用dynamodb local//docs.aws.amazon.com/amazondynamodb/latest/developerguide/dynamodblocal.html
从以下位置获取 dynamodb: ://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.DynamoDBLocal.html 然后运行: java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar 卡夫卡...
terraform-aws-dynamodb-autoscaler 用于配置DynamoDB自动缩放器的Terraform模块。 Autoscaler会根据负载为DynamoDB表扩展/缩减已配置的OPS。 该项目是我们针对DevOps的全面方法的一部分。 它是100%开源的,并根据...
java8集合源码无服务器架构 Trabalho de Serverless 架构为 ...//c/databases/dynamodb/trip/dynamodb:/data/amazon/dynamodb-local -jar DynamoDBLocal.jar -sharedDb -dbPath /data ### 在 DynamoDB
dynamodb-document-js-sdk 抽离出低端 SDK 属性值的类型,以提供更简单的开发经验。JS 数据类型,如 string 和 number 可以直接传送到 DynamoDB 需求中;同样,数据类型不会被打包。示例代码:// Basic Client ...
Spring Webflux,AWS SDK,Swagger,DynamoDB,JUnit 启用AWS Access Key安装AWS CLI Open终端并输入:aws configure 在以下链接中下载本地DynamoDB //docs.aws.amazon..../ DynamoDBLocal_lib -jar DynamoDBLocal.jar ...
DynamoDB是亚马逊提供的完全托管的NoSQL数据库服务,允许用户无需管理服务器即可存储和检索任意量的数据。DynamoDB支持多种数据类型,包括标量值、文档和集合类型。它能够为数据提供快速的访问速度,并且能够自动...
在本文中,我们将深入探讨如何在Laravel框架中利用DynamoDB进行开发,基于提供的标题“Laravel开发-dynamodb”和描述中提到的资源“laravel-dynamodb”库。DynamoDB是Amazon Web Services(AWS)提供的一款无服务器...
查询和扫描(Query and Scan): 可以根据特定条件执行查询操作,或者扫描整个表以查找数据。 4. 使用二级索引(Secondary Indexes): DynamoDB允许在表上创建二级索引,以便能够更高效地查询数据。二级索引分为...
dynamodb- //console.aws.amazon.com/dynamodb/home region us-east-1#tables:selected dataTable 策略编辑-https: us- -1#/ arn: policy/ 日志流-https: us- -1# logStream:group=/ aws/lambda/ ...
DynamoDB官方文档,入门文档。学习dynamoDB的工具,开发人员指南,API。 Amazon DynamoDB 是一种完全托管的 NoSQL 数据库服务,提供快速且可预测的性能,同时还能够实现无缝扩展。使用 DynamoDB,您可以免除操作和...
- [Amazon DynamoDB 开发者指南](http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/operationlist.html) **扩展知识**: - DynamoDB 是一种高性能、完全托管的 NoSQL 数据库服务,适用于 Web 和...
DynamoDB Cookbook by Tanmay Deshpande, Over 90 hands-on recipes to design Internet scalable web and mobile applications with Amazon DynamoDB
发电机续集请勿使用 一个支持Sequelize和Dynamodb(受限)的sequelize包装器。用import Sequelize from 'sequelize'import SequelizeDynamo from 'dynamo-sequelize'// Only when dialect === 'dynamo', will use ...
使用DynamoDB的Java / Maven中的无服务器REST API 示例无服务器服务将为产品创建REST API。 它将部署到AWS。 数据将存储在DynamoDB表中。 这是博客文章的配套应用程序。 安装先决条件 node和npm 从安装JDK而不是...
在本文中,我们将深入探讨如何在Laravel的微服务框架Lumen中,集成一个文件管理器并使用Amazon DynamoDB作为存储后端。Lumen是一款轻量级、高性能的PHP框架,适合构建API和微服务应用。DynamoDB是亚马逊云服务(AWS...
Query , Table , } from "@serverless-seoul/dynamorm" ; @ Decorator . Table ( { name : `your_table_name_on_aws_dynamodb` } ) export class BlogPost extends Table { @ Decorator . HashPrimaryKey ( "id" ...
《Go-dynago:Go语言中的DynamoDB客户端详解》 在现代的云原生应用开发中,Amazon DynamoDB作为一款高性能、完全托管的NoSQL数据库服务,被广泛应用于存储和检索大规模数据。针对Go语言的开发者,亚马逊AWS提供了一...
在本文中,我们将深入探讨如何在Laravel框架中使用DynamoDB作为会话驱动程序,以便在AWS(亚马逊网络服务)环境中实现高效、可扩展的会话管理。Laravel是PHP的一个流行开源Web应用程序框架,它提供了丰富的功能来...