Amazon DynamoDB is NoSQL database. It supports ACID – atomicity, consistency, isolation, and durability transactions.
DynamoDB is a stateless web service; maintaining persistent network connections is not needed. Instead, operations with DynamoDB occur using HTTP(S) requests and responses.
Designing for DynamoDB:
In DynamoDB, “locality of reference” is key concept in designing tables; it is anti-normalization.
De-normalization by embedding child entities like customer address as L (List) datatype may be considered.
Before designing, it is essential to understand business need, workflow and query patterns and properties of query patterns like data size, data shape and data velocity
To balance performance and cost, it would be wise to consider Read Consistency Types and Consistency parameter and On-Demand Mode and Provisioned Mode and Reserved Capacity.
Designing partition key is very important for good performance. DynamoDB uses partition key and its internal hash function to find the partition to write and read data; if sort key is provided, value for the sort key is stored in ascending order. For Customer table, customer city may be partition key, customer name may be sort key, if your query pattern reflects more city wise search.
If ProvisionedThroughputExceededException is thrown, it may be due to that query pattern is inconsistent with partition key.
Following are the core parts of DynamoDB
Table (Local and Global – collection of items) Attribute (fields)
Partition Key (primary key) – may be single or composite Partition Key and sort key (Composite primary key)
One or more Secondary indexes (5 Local and 20 Global per table – speed up reads) Streams (new event for item create, update or delete)
Global Tables for replication across regions of choice (read and write locally and access globally)
It facilitates access from AWS console, CLI and SDK.
List of data types:
S – String, N – Number, B – Binary, BOOL – Boolean, NULL – Null, M – Map, L – List, SS – String Set, NS – Number Set, BS – Binary Set *
* DynamoDB uses JSON only as a transport protocol, not as a storage format. JSON does not natively support encoding binary data. So, you will need to encode binary data in base64 format. DynamoDB decodes the base64 data back to binary.
It exposes its methods via web services as part of SDK. The SDK supports add, update (update in place), delete, query, etc. The SDK supports many popular programming languages. For Java and Dot Net, they provide objects for table items with annotation like @DynamoDBTable.
Working with queries is different from RDBMS. For search criteria, we use condition expression on partition key with value; optionally second condition (=, >, >=, <, <=, BETWEEN and beginswith) can be applied on sort key. On Query results, filters can be applied with filter expression that cannot use partition key or sort key. Filter expressions can use the same comparators, functions, and logical operators as a condition expression, with the addition of the not-equals operator (<>).
The Query response contains the following elements also:
ScannedCount — number of items that matched the key condition expression before applying filter Count — number of items that remain after applying filter
It is similar to RDBMS table scan
Items (records) are not locked during a transaction as in RDBMS. DynamoDB transactions provide serializable isolation. Transactional operations work within one or more tables within a region, and are not supported across regions in global tables.
We can summarize the isolation levels for various transaction operations PutItem – serializable,
DeleteItem – serializable
UpdateItem – serializable GetItem – serializable Query – serializable
Scan – serializable
BatchWriteItem – Not serializable – individual operations are serializable BatchGetItem – Read Commited – individual operations are serializable
Two DynamoDB operations have been introduced for handling multiple write/read operations in single transactions. It is similar to stored procedure or function in RDBMS.
- TransactWriteItems – a batch that can contain a write set, with one or
more PutItem, UpdateItem, and DeleteItem with optional check for prerequisite conditions that must be satisfied before making updates. If any condition is not met, the transaction is rejected.
- TransactGetItems – a batch that can contain a read set, with one or more If
a TransactGetItems request is issued on an item that is part of an active write transaction, the read transaction is cancelled. To get the previously committed value, you can use a standard read.
– Kuppuram, CTO, We3cares