-
Notifications
You must be signed in to change notification settings - Fork 9.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ggml : improve ggml_is_contiguous logic #7856
Conversation
This seems very unlikely, but this check would detect a case such as this as contiguous, right? type = GGML_TYPE_F32
ne = ([0] = 2, [1] = 1, [2] = 1, [3] = 2)
nb = ([0] = 4, [1] = 8, [2] = 9999, [3] = 9999) |
How about something like this (untested): static bool ggml_is_contiguous_n(const struct ggml_tensor * tensor, int n) {
size_t next_nb = ggml_type_size(tensor->type);
if (tensor->ne[0] != ggml_blck_size(tensor->type) && tensor->nb[0] != next_nb) {
return false;
}
next_nb *= tensor->ne[0]/ggml_blck_size(tensor->type);
for (int i = 1; i < GGML_MAX_DIMS; i++) {
if (tensor->ne[i] != 1) {
if (i > n) {
if (tensor->nb[i] != next_nb) {
return false;
}
next_nb *= tensor->ne[i];
}
else {
// this dimension does not need to be contiguous
next_nb = tensor->ne[i]*tensor->nb[i];
}
}
}
return true;
}
GGML_CALL bool ggml_is_contiguous(const struct ggml_tensor * tensor) {
return ggml_is_contiguous_0(tensor);
}
GGML_CALL bool ggml_is_contiguous_0(const struct ggml_tensor * tensor) {
return ggml_is_contiguous_n(tensor, 0);
}
GGML_CALL bool ggml_is_contiguous_1(const struct ggml_tensor * tensor) {
return ggml_is_contiguous_n(tensor, 1);
}
GGML_CALL bool ggml_is_contiguous_2(const struct ggml_tensor * tensor) {
return ggml_is_contiguous_n(tensor, 2);
} |
Good catch - will revise tomorrow |
@slaren I implemented your suggestion. Should be ready to merge |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, I also verified that the assembly generated is very similar for both versions, so I don't expect this implementation to be slower. I only ran very limited tests though, and the logic is not very easy to follow for me, so I am not 100% sure that it will work in every case.
With this change, tensors with the following dimensions will now be considered contiguous:
The logic being that if a dimension
ne[d] == 1
then the stride is allowed to be arbitrary because we will never use it to offset a ptr (i.e.id * nb[d]
will always be0
becauseid
loops in [0, 1), i.e. it is always 0)This needs careful review as I'm not 100% confident about the impact. Started looking into this after trying to assert correctly the unary op implementations in #7857