`
zzc1684
  • 浏览: 1229449 次
  • 性别: Icon_minigender_1
  • 来自: 广州
文章分类
社区版块
存档分类
最新评论

use batches to delete large number of records in a table

阅读更多

Sometimes it can happen that you need to delete a lot of records in a table. This can happen for instance in log tables. With a simple DELETE statement like:

DELETE FROM MyTable WHERE COL1 = 1

it can take a while when you have for instance 1 million records to delete. It can results in a table lock which has a negative impact on the performance of your application.

As of SQL2005/2008 you can delete records in a table in batches with the DELETE TOP (BatchSize) statement. This method has 3 advantages

  1. It will not create one big transaction.
  2. It avoids a table lock.
  3. If the delete statement is canceled, only the last batch is rolled back. Records in previous batches are deleted.

You can use next script to test this DELETE TOP () statement.

CREATE TABLE DEMO (COL1 INT,COL2 INT)


DECLARE @COUNTER INT
SET @COUNTER = 1

INSERT INTO DEMO (COL1,COL2) Values (2,2)

WHILE @COUNTER < 50000
BEGIN

INSERT INTO DEMO (COL1,COL2) Values (1,@COUNTER)
SET @COUNTER = @COUNTER + 1

END

/*
-- Show content of the table
SELECT COL1, COUNT(*) FROM DEMO GROUP BY COL1
*/

-- Deleting records in batches of 1000 records

DECLARE @BatchSize INT
SET @BatchSize = 1000

WHILE @BatchSize <> 0

BEGIN

DELETE TOP (@BatchSize)
FROM DEMO
WHERE COL1 = 1

SET @BatchSize = @@rowcount

END

-- SELECT * FROM Demo -- Now we have only 1 record left

分享到:
评论

相关推荐

    微软内部资料-SQL性能优化5

    On a qualified select, update, or delete, the correct leaf page will be the lowest page of the tree in which one or more rows with the specified key or keys reside. A qualified operation is one that ...

    DON’T USE LARGE MINI-BATCHES, USE LOCAL SGD.pdf

    本文档标题所提到的“DON’T USE LARGE MINI-BATCHES, USE LOCAL SGD”指向的是关于分布式训练中使用局部随机梯度下降(Stochastic Gradient Descent,SGD)的一个重要观点。传统的SGD方法在处理大规模深度神经网络...

    微软内部资料-SQL性能优化3

    Used to lock ranges between records in a table to prevent phantom insertions or deletions into a set of records. Ensures serializable transactions. RID A Row Identifier. Used to individually lock a ...

    clickhouse_en(1).pdf

    For reads, quite a large number of rows are extracted from the DB, but only a small subset of columns. Tables are “wide,” meaning they contain a large number of columns. Queries are relatively rare ...

    delete_in_batches:快速批量删除Active Record和Postgres

    delete_in_batches ... delete_in_batches ( batch_size : 50000 ) # defaults to 10000 批次之间睡眠 Tweet . where ( user_id : 1 ) . delete_in_batches ( sleep : 0.01 ) 显示进度 Tweet . where ( user_id :

    SQL Server 2014 Development Essentials - Masood-Al-Farooq, Basit A. [SRG].pdf

    This chapter covers how to add data to a table using the INSERT statement, how to delete the data using the DELETE statement, and how to update existing data using the UPDATE statement. This chapter...

    计算机网络第六版答案

    Wifi (802.11) In a wireless LAN, wireless users transmit/receive packets to/from an base station (i.e., wireless access point) within a radius of few tens of meters. The base station is typically ...

    in_batches:ActiveRecord的反向移植

    针对Rails 4应用程序的ActiveRecord::Relation#...)Person.in_batches.update_all(awesome: true)Person.in_batches.delete_allPerson.in_batches(of: 10_000).map do |relation| relation.delete_all sleep 10 # Throt

    Pro Spark Streaming,The Zen of Real-time Analytics using Apache Spark

    On the Spark Streaming front, two major features have been added: mapWithState to maintain state across batches and using back pressure to throttle the input rate in case of queue buildup.2 In ...

    On Multihop Communications For In-Vehicle Internet Access Based

    A vehicular ad hoc network (VANET) is an emerging technology which has a great potential in realizing a variety of new applications...are served in batches of no more than a specified maximum batch-size.

    Learning Apache Cassandra - Second Edition

    His experience includes handling large amounts of unstructured and structured data in Hadoop, and developing data processing applications using Spark and MapReduce. Right now, he is working with some ...

    Applied.Multivariate.Statistical.Analysis.4th.Edition.366245170

    All of these examples involve high to ultra-high dimensions and represent a number of major fields in big data analysis. The fourth edition of this book on Applied Multivariate Statistical Analysis ...

    ActiveXperts SMS and Pager Toolkit3.2_crack

    To send out SMS messages in batches, to notify groups of people; To receive SMS messages and store them in a database, or reply them; To control your mobile phone with your PC, adding phonebook ...

    Flow-through cell-based In Vitro Release Method for Triamcinolone acetonide

    The main objective of the current research was to develop a compendial flow-through cell apparatus based in vitro release testing method for sustained-release triamcinolone acetonide–loaded poly ...

    cifar-10-batches-py.zip

    The CIFAR-10 dataset consists of 60000 32x32 colour images in 10 classes, with 6000 images per class. There are 50000 training images and 10000 test images. The dataset is divided into five training...

    Salesforce doc: Integration Patterns and Practices

    **Pattern Summary**: This pattern addresses the synchronization of data between Salesforce and an external system in batches. It is ideal for scenarios where large volumes of data need to be ...

    万能脱壳工具 病毒分析工具 V1.4

    you should get the warranty and permission of DSWLAB before(for example, if the anti-virus company want to use it to analyses the Trojan horse in batches, he must get mandate and permission of DSWLAB...

    Modern Operating Systems 3rd

    9. **The World According to C:** The book explores the use of the C programming language in operating system development. Topics include language features, header files, large programming projects, ...

Global site tag (gtag.js) - Google Analytics