IPAdapter + Sliding window in animatediff is not working

[Gutianpei]

Describe the bug

The weights of ipadapter can be correctly applied! Thanks for the fix! However, I found that the sliding window in animatediff is not working with ipadapter's weight, and it seems the sub_idxs below never gets updated.

https://github.com/siliconflow/onediff/blob/main/onediff_comfy_nodes/modules/oneflow/infer_compiler_registry/register_comfy/CrossAttentionPatch.py#L108

Your environment

Ubuntu 20.04

OneDiff git commit id

2249010

OneFlow version info

libibverbs not available, ibv_fork_init skipped
path: ['/home/ubuntu/miniconda3/envs/comfy/lib/python3.10/site-packages/oneflow']
version: 0.9.1.dev20240420+cu121
git_commit: 665bcf8
cmake_build_type: Release
rdma: True
mlir: True
enterprise: False

How To Reproduce

The workflow below can reproduce the bug.
onediff_adiff_ipadapter_reproduce.json

The complete error message

No error message noted.

Additional context

Dependency version is followed this pr: #879
ComfyUI-AnimateDiff-Evolved commit id: f9e0343f4c4606ee6365a9af4a7e16118f1c45e1