Post by Loic Dachary Post by Thomas Goirand Post by Loic Dachary Post by James Page
I dug into this in a bit more detail today; the Ceph package builds a
number of difference loadable erasure coding plugins, enabling
different cpu feature sets (generic, neon, sse3, sse4); each time
Jerasure and gf-complete are statically linked into the module, built
with the required flags to enable the right CPU instruction codes
(build time, not runtime enablement).
Yes, and depending on which CPU features are available at runtime,
the plugin that can take advantage of them is loaded. All variants
are verified to encode / decode exactly in the same way (with a set
of data) each time a new Ceph version is published, to protect
against regression or corruption.
Such a feature should be upstreamed in gf-complete and jerasure.
Post by Loic Dachary Post by James Page
Unless I'm reading the packaging wrong, the jerasure and gf-complete
packages current disable any CPU specific extensions in order to have
a completely generic library that works on any CPU. So using the
system libraries effectively cripples any CPU optimization that might
be achievable at runtime in Ceph.
Yes. I could fix that without sacrificing test coverage / data
integrity, simply by moving part of the above in the package, and
running integration and non regression tests on the resulting package.
The package could be used in the current Ceph integration suites,
before being uploaded to Debian GNU/Linux. That's the most effective
way to protect jerasure users against data loss right now.
Could you please share your patches and open a bug against the
gf-complete and jerasure packages?
I'm offended by the mails you sent to me on that topic. They were
disrespectful, condescending and uncooperative. Given that's the
first and only time you chommunicated with me since you started
packaging jerasure, it would be wise of you to hand over the package
to someone better able to establish a productive and friendly
discussion with upstream.
Well, disrespect is what you get when you are disrespectful yourself.
Asking to hand-over the package for no valid reason, with no bug opened
at all, after all the effort and time I spent on these was really
disrespectful indeed. You still haven't opened any bug btw, please do so
if you have any concern.
Now I fail to see how uncooperative I have been. I've been here asking
you to share your work within the packages of gf-complete, and jerasure,
but it doesn't seem like you want to. In what way is this an
uncooperative attitude from my side? Just because I refuse to hand-over
the packages to you (only, without a team)? Or is this because I've told
you that it should be upstream to do runtime CPU feature detection? The
only answer I get is that I'm "disrespectful, condescending and
uncooperative" in return. This isn't going in the right direction.
I by the way would like to point out that you are still a member of the
OpenStack packaging group on Alioth, where all of these packages are
maintained. So if you would like to contribute, the git repositories are
wide open to you. I don't understand why I would have to just give-up
the packages to you and you only. No, these packages are *not* getting
away from the OpenStack packaging group. Yes, you can help or even take
care of them if you wish. I've always been very welcoming contributors,
but we need to keep this kind of key package in the group to make sure
they are well integrated with everything else in OpenStack.
In the past, communicating with you was done in a very friendly way. I
don't understand what happened suddenly. What I feel from my side is
aggressiveness from yours. So I propose that we both forget it, and try
to reboot all. If I offended you, I'm sorry about it. I hope you'll
forget it, and I'll try to forgive you too.
Shall we move on? Can we have a constructive conversation from now on?
Let me do another attempt, and see how it goes: what's your plan for the
packages to do CPU feature detection at runtime, and get Ceph to use
that, instead of embedding the libs?
I also have plans myself, btw, but I was lacking time. I know there's
some ways to add CPU features built in libs (dropped in specific
folders). Another way would be to have another binary including the
optimizations (which we could call for example libgf-complete1-sse4, and
which would Provides: and Breaks: libgf-complete1).
I'd prefer the first approach, but I never had time to investigate how
it's done in Debian. There's also the issue that some buildd may not
have the SSE4 features, and therefore, they may not be able to build the
packages. This would have to be investigated too.
I'm looking forward to read your answer, then let's see together how it
Thomas Goirand (zigo)
To UNSUBSCRIBE, email to firstname.lastname@example.org
with a subject of "unsubscribe". Trouble? Contact ***@lists.debian.org