#822 mod_http_upload (and many other places): replace UUIDs with shorter identifiers
Reporter
Ge0rG
Owner
Zash
Created
Updated
Stars
★★ (3)
Tags
Priority-Low
Status-Fixed
Type-Enhancement
Ge0rG
on
Prosody is using `util.uuid.generate()` all over the place. While this is reasonable in internal structures, it adds overhead on the networks when exposed externally (resource identifiers, message IDs, http_upload URLs).
With URLs this is especially severe because they are often communicated out-of-band, displayed in text-oriented XMPP clients or copied into emails.
The image URL I got from http_upload on yax.im (via xmpp.yaxim.org) is over 100 characters long, containing two 36-character UUIDs.
Please replace the UUID generator with a function that provides shorter IDs by using more of the available bits, e.g. url-safe-base64(uuid()) or whatever other encoding provides the best results.
Zash
on
Would be nice.
Tricky to do without either breaking 0.9 support or adding extra code in the modules. I'm not sure either is worth it for saving ~12 characters.
Could use util.uuid.get_nibbles(), which returns an arbitrary number of hex digits. 128 bits is probably overkill in many cases.
Changes
tags Priority-Low Status-Accepted
Ge0rG
on
Wouldn't it be possible to add a new `gen_short()` function in 0.10+, and have the module instanciate it as `util.uuid.gen_short or util.uuid.generate`?
The hex encoding is really what bothers me the most about this, and I'm not sure how far we can go down from 128 bit without getting nasty birthday attacks or evil twins or I don't know.
My prosody 0.9.12 returns an error trying to use conversations.im "send image" when calling the new get_nibbles() function:
Jan 30 20:17:36 mod_c2s error Traceback[c2s]: /opt/prosody-modules/mod_http_upload/mod_http_upload.lua:90: attempt to call upvalue 'random_hex' (a nil value)
stack traceback:
/opt/prosody-modules/mod_http_upload/mod_http_upload.lua:90: in function '?'
/usr/lib64/prosody/util/events.lua:67: in function </usr/lib64/prosody/util/events.lua:63>
(tail call): ?
/usr/lib64/prosody/modules/mod_iq.lua:70: in function '?'
/usr/lib64/prosody/util/events.lua:67: in function 'fire_event'
/usr/lib64/prosody/core/stanza_router.lua:187: in function 'core_post_stanza'
/usr/lib64/prosody/core/stanza_router.lua:135: in function </usr/lib64/prosody/core/stanza_router.lua:56>
(tail call): ?
[C]: in function 'xpcall'
/usr/lib64/prosody/modules/mod_c2s.lua:123: in function 'cb_handlestanza'
...
/usr/lib64/prosody/util/xmppstream.lua:255: in function 'feed'
/usr/lib64/prosody/modules/mod_c2s.lua:233: in function 'data'
/usr/lib64/prosody/modules/mod_c2s.lua:255: in function 'onincoming'
/usr/lib64/prosody/net/server_event.lua:636: in function </usr/lib64/prosody/net/server_event.lua:582>
[C]: in function 'loop'
/usr/lib64/prosody/net/server_event.lua:797: in function </usr/lib64/prosody/net/server_event.lua:796>
[C]: in function 'xpcall'
/usr/lib64/prosody/../../bin/prosody:376: in function 'loop'
/usr/lib64/prosody/../../bin/prosody:407: in main chunk
[C]: ?
Is this supposed to be incompatible with 0.9 or do I have to update something else to get this function?
Zash
on
Turns out it wasn't a public function after all. I suppose I'll reverse this then.
ageru
on
Hi Zash,
Just to confirm the issue Pascal reported was fixed by your revert. Many thanks!
For the record, I'm with 0.9.12 on x86 and this is what prosody.err logged before I pulled your fix:
stack traceback:
...bber/modules-enabled/mod_http_upload/mod_http_upload.lua:90: in function <...bber/modules-enabled/mod_http_upload/mod_htt
p_upload.lua:58>
/usr/lib64/prosody/util/events.lua:67: in function 'fire_event'
/usr/lib64/prosody/modules/mod_iq.lua:70: in function </usr/lib64/prosody/modules/mod_iq.lua:62>
/usr/lib64/prosody/util/events.lua:67: in function 'fire_event'
/usr/lib64/prosody/core/stanza_router.lua:187: in function 'core_post_stanza'
/usr/lib64/prosody/core/stanza_router.lua:135: in function </usr/lib64/prosody/core/stanza_router.lua:56>
[C]: in function 'cb_handlestanza'
/usr/lib64/prosody/util/xmppstream.lua:187: in function </usr/lib64/prosody/util/xmppstream.lua:167>
[C]: in function 'parse'
/usr/lib64/prosody/util/xmppstream.lua:255: in function 'feed'
/usr/lib64/prosody/modules/mod_c2s.lua:233: in function 'data'
/usr/lib64/prosody/modules/mod_c2s.lua:255: in function 'onincoming'
/usr/lib64/prosody/net/server_event.lua:636: in function </usr/lib64/prosody/net/server_event.lua:582>
[C]: in function 'loop'
/usr/lib64/prosody/net/server_event.lua:797: in function </usr/lib64/prosody/net/server_event.lua:796>
[C]: in function 'xpcall'
/usr/lib64/prosody/../../bin/prosody:376: in function 'loop'
/usr/lib64/prosody/../../bin/prosody:407: in main chunk
[C]: at 0x0804b920
Kind regards,
ageru
This [fix](https://hg.prosody.im/prosody-modules/rev/45ef16ebe565) broke upload on Converstations and Gajim for me! Had to revert the change.
Using Prosody 0.10 nightly build 369
Am I missing something? Do I need to change the prosody.conf.lua?
evildead
on
Hi @ all,
I had the same issue with Conversations and Gajim. I changed line 24 of file mod_http_upload/mod_http_upload.lua:
- function uuid()
+ local function uuid()
Now it works for me.
Best regards,
evildead
Prosody is using `util.uuid.generate()` all over the place. While this is reasonable in internal structures, it adds overhead on the networks when exposed externally (resource identifiers, message IDs, http_upload URLs). With URLs this is especially severe because they are often communicated out-of-band, displayed in text-oriented XMPP clients or copied into emails. The image URL I got from http_upload on yax.im (via xmpp.yaxim.org) is over 100 characters long, containing two 36-character UUIDs. Please replace the UUID generator with a function that provides shorter IDs by using more of the available bits, e.g. url-safe-base64(uuid()) or whatever other encoding provides the best results.
Would be nice. Tricky to do without either breaking 0.9 support or adding extra code in the modules. I'm not sure either is worth it for saving ~12 characters. Could use util.uuid.get_nibbles(), which returns an arbitrary number of hex digits. 128 bits is probably overkill in many cases.
ChangesWouldn't it be possible to add a new `gen_short()` function in 0.10+, and have the module instanciate it as `util.uuid.gen_short or util.uuid.generate`? The hex encoding is really what bothers me the most about this, and I'm not sure how far we can go down from 128 bit without getting nasty birthday attacks or evil twins or I don't know.
mod_http_upload change in https://hg.prosody.im/prosody-modules/rev/f48e9e25aec4
ChangesMy prosody 0.9.12 returns an error trying to use conversations.im "send image" when calling the new get_nibbles() function: Jan 30 20:17:36 mod_c2s error Traceback[c2s]: /opt/prosody-modules/mod_http_upload/mod_http_upload.lua:90: attempt to call upvalue 'random_hex' (a nil value) stack traceback: /opt/prosody-modules/mod_http_upload/mod_http_upload.lua:90: in function '?' /usr/lib64/prosody/util/events.lua:67: in function </usr/lib64/prosody/util/events.lua:63> (tail call): ? /usr/lib64/prosody/modules/mod_iq.lua:70: in function '?' /usr/lib64/prosody/util/events.lua:67: in function 'fire_event' /usr/lib64/prosody/core/stanza_router.lua:187: in function 'core_post_stanza' /usr/lib64/prosody/core/stanza_router.lua:135: in function </usr/lib64/prosody/core/stanza_router.lua:56> (tail call): ? [C]: in function 'xpcall' /usr/lib64/prosody/modules/mod_c2s.lua:123: in function 'cb_handlestanza' ... /usr/lib64/prosody/util/xmppstream.lua:255: in function 'feed' /usr/lib64/prosody/modules/mod_c2s.lua:233: in function 'data' /usr/lib64/prosody/modules/mod_c2s.lua:255: in function 'onincoming' /usr/lib64/prosody/net/server_event.lua:636: in function </usr/lib64/prosody/net/server_event.lua:582> [C]: in function 'loop' /usr/lib64/prosody/net/server_event.lua:797: in function </usr/lib64/prosody/net/server_event.lua:796> [C]: in function 'xpcall' /usr/lib64/prosody/../../bin/prosody:376: in function 'loop' /usr/lib64/prosody/../../bin/prosody:407: in main chunk [C]: ? Is this supposed to be incompatible with 0.9 or do I have to update something else to get this function?
Turns out it wasn't a public function after all. I suppose I'll reverse this then.
Hi Zash, Just to confirm the issue Pascal reported was fixed by your revert. Many thanks! For the record, I'm with 0.9.12 on x86 and this is what prosody.err logged before I pulled your fix: stack traceback: ...bber/modules-enabled/mod_http_upload/mod_http_upload.lua:90: in function <...bber/modules-enabled/mod_http_upload/mod_htt p_upload.lua:58> /usr/lib64/prosody/util/events.lua:67: in function 'fire_event' /usr/lib64/prosody/modules/mod_iq.lua:70: in function </usr/lib64/prosody/modules/mod_iq.lua:62> /usr/lib64/prosody/util/events.lua:67: in function 'fire_event' /usr/lib64/prosody/core/stanza_router.lua:187: in function 'core_post_stanza' /usr/lib64/prosody/core/stanza_router.lua:135: in function </usr/lib64/prosody/core/stanza_router.lua:56> [C]: in function 'cb_handlestanza' /usr/lib64/prosody/util/xmppstream.lua:187: in function </usr/lib64/prosody/util/xmppstream.lua:167> [C]: in function 'parse' /usr/lib64/prosody/util/xmppstream.lua:255: in function 'feed' /usr/lib64/prosody/modules/mod_c2s.lua:233: in function 'data' /usr/lib64/prosody/modules/mod_c2s.lua:255: in function 'onincoming' /usr/lib64/prosody/net/server_event.lua:636: in function </usr/lib64/prosody/net/server_event.lua:582> [C]: in function 'loop' /usr/lib64/prosody/net/server_event.lua:797: in function </usr/lib64/prosody/net/server_event.lua:796> [C]: in function 'xpcall' /usr/lib64/prosody/../../bin/prosody:376: in function 'loop' /usr/lib64/prosody/../../bin/prosody:407: in main chunk [C]: at 0x0804b920 Kind regards, ageru
https://hg.prosody.im/prosody-modules/rev/45ef16ebe565
ChangesThis [fix](https://hg.prosody.im/prosody-modules/rev/45ef16ebe565) broke upload on Converstations and Gajim for me! Had to revert the change. Using Prosody 0.10 nightly build 369 Am I missing something? Do I need to change the prosody.conf.lua?
Hi @ all, I had the same issue with Conversations and Gajim. I changed line 24 of file mod_http_upload/mod_http_upload.lua: - function uuid() + local function uuid() Now it works for me. Best regards, evildead