hibernate 注解 manytomany 级联删除问题

定义了两个类，Account 和 Department。是多对多关系，类定义如下;
@Entity
@Table(name = "ACCOUNTS")
@Cache(usage = CacheConcurrencyStrategy.READ_ONLY)
public class Account {
        private Long id;
private String loginName;
private List<Department> departments;
...

@Override
@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "SEQ_STORE")
@SequenceGenerator(name = "SEQ_STORE", sequenceName = "ACCOUNTS_ID", allocationSize = 2)
@Column(length = 12)
public Long getId() {
return id;
}

public void setId(Long id) {
this.id = id;
}
@ManyToMany(fetch = FetchType.LAZY)
@OrderBy("priority desc")
@JoinTable(name = "ACCOUNT_DEPARTMENT_ACCOUNT", joinColumns = { @JoinColumn(name = "account_id", nullable = false, updatable = false) }, inverseJoinColumns = { @JoinColumn(name = "department_id", nullable = false, updatable = false) })
@Cascade( { CascadeType.SAVE_UPDATE })
public List<Department> getDepartments() {
return departments;
}

public void setDepartments(List<Department> departments) {
this.departments = departments;
}
}

@Entity
@Table(name = "ACCOUNT_DEPARTMENTS")
@Cache(usage = CacheConcurrencyStrategy.READ_ONLY)
public class Department {
private Long id;

private List<Department> children;
private List<Account> accounts;

@Override
@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "SEQ_STORE")
@SequenceGenerator(name = "SEQ_STORE", sequenceName = "ACCOUNT_DEPARTMENTS_ID", allocationSize = 2)
@Column(length = 12, name = "ID")
public Long getId() {
return id;
}

public void setId(Long id) {
this.id = id;
}

@OneToMany(fetch = FetchType.LAZY)
@OrderBy("priority desc")
@Cascade(CascadeType.ALL)
@JoinColumn(name = "parentId")
public List<Department> getChildren() {
return children;
}

public void setChildren(List<Department> children) {
this.children = children;
}
@ManyToMany(fetch = FetchType.LAZY)
@OrderBy("priority desc")
@JoinTable(name = "ACCOUNT_DEPARTMENT_ACCOUNT", joinColumns = { @JoinColumn(name = "department_id", nullable = false, updatable = false) }, inverseJoinColumns = { @JoinColumn(name = "account_id", nullable = false, updatable = false) })
@Cascade( { CascadeType.SAVE_UPDATE })
public List<Account> getAccounts() {
return accounts;
}

public void setAccounts(List<Account> accounts) {
this.accounts = accounts;
}
}
自动在数据库中生成三个表 ACCOUNTS,ACCOUNT_DEPARTMENTS,和中间表ACCOUNT_DEPARTMENT_ACCOUNT。
现在我删除部门表(ACCOUNT_DEPARTMENTS)的一个记录时，（假设id=3删除），它会把中间表ACCOUNT_DEPARTMENT_ACCOUNT中的 department_id = 3的记录删除。
还会把ACCOUNT_DEPARTMENT_ACCOUNT中 department_id = 3 对应的人员表记录id，例如是account_id = 4，account_id = 5，把ACCOUNTS中的id为4，和5的记录也会删掉。
同时还会把 ACCOUNT_DEPARTMENTS 中id=3，和子部门 = 3 的记录也删掉。

我是想只删掉中间表中 department_id = 3，和 ACCOUNT_DEPARTMENTS 中id = 3的记录。
请问这样该怎么实现呢？