DynamoDB and Query/Scan Limit

DynamoDB and Query/Scan Limit

Recently, I found one issue in my Project for DynamoDB.
I do query by index and some filters. But even I put
SELECT: “COUNT”

DynamoDB will only return me parts of the whole records. It is said the limit is 4M. Even the count is part of the whole results.

I need to do pagination and add up all the count together to get the final total count. Or use a list to hold the Items and add up all the Items to get the while results.

Here is the example of query with pagination

export const countCurrentMeetingsByGroup = async (groupUUID: string) => {
    const tempMeeting = false;
    let lastEvaluatedKey = null;
    let countTotal = 0;
    do {
        const params: any = {
            TableName: meetingTableName,
            IndexName: meetingTableIndex,
            KeyConditionExpression: "#groupUUID = :groupUUID",
            FilterExpression: "#tempMeeting = :tempMeeting AND #action <> :action ",
            ExpressionAttributeNames:{
                "#groupUUID": "groupUUID",
                "#tempMeeting": "tempMeeting",
                "#action": "action"
            },
            ExpressionAttributeValues: {
                ":groupUUID": groupUUID,
                ":tempMeeting": tempMeeting,
                ":action": "DELETED"
            },
            Select: 'COUNT'
        };

        if (lastEvaluatedKey) {
            params['ExclusiveStartKey'] = lastEvaluatedKey;
        }

        console.log('getting countCurrentMeetingsByGroup from DB with params: ', params);
        const data = await dynamoDb.query(params).promise();
        lastEvaluatedKey = idx(data, _ => _.LastEvaluatedKey);
        const count = idx(data, _ => _.Count);
        console.log(`data=${JSON.stringify(data)}`);
        console.log(`lastEvaluatedKey=${JSON.stringify(lastEvaluatedKey)}`);
        console.log(`count=${count}`);
        if (count) {
            countTotal = countTotal + count;
        }
        console.log('countCurrentMeetingsByGroup load from DB, meeting count = ' + count + ' countTotal = ' + countTotal);
    } while (lastEvaluatedKey);
    return countTotal;
};

The most important part is

lastEvaluatedKey = idx(data, _ => _.LastEvaluatedKey);


References:
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-Select
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Query.html#Query.Limit
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html#DDB-Query-request-Select
https://egkatzioura.com/2016/07/20/query-dynamodb-items-with-node-js-part-2/
https://stackoverflow.com/questions/15543418/how-to-get-the-result-of-describetable-amazon-dynamodb-method-in-node-js
https://github.com/awslabs/dynamodb-data-mapper-js/issues/72