#1082 util.pubsub: Node config cached in storage abstraction and not updated
Reporter
Ivan Vučica
Owner
Zash
Created
Updated
Stars
★ (1)
Tags
Priority-Medium
Status-Fixed
Type-Defect
Milestone-0.11
Ivan Vučica
on
In mod_pep_plus, I've created multiple entries in urn:xmpp:microblog:0. By inspecting the PostgreSQL database, I can see that the items are there.
However the following seems to fetch only one item.
<iq type='get'
to='my@bare.jid'
id='items3'>
<pubsub xmlns='http://jabber.org/protocol/pubsub'>
<items node='urn:xmpp:microblog:0'/>
</pubsub>
</iq>
Same if I set max_items="5" (as an experiment).
If I delete an item, the next one rears its head. If I fetch by a specific ID, I get the older items from the database, too.
This is on Docker image based on Ubuntu Zesty. This is from Prosody trunk: 0de0018bdf91
As far as I can tell, mod_pep_plus is now only in trunk, so that's the version used.
Ivan Vučica
on
What I failed to mention is that yes, I have increased max_items on the node and confirmed that its persist_items is true :-)
Zash
on
Did you check the SQL executed with mod_debug_sql?
If you could try with internal storage too, that would be helpful to figure out what limits it to one item.
Changes
tags Status-Accepted Milestone-0.11
Ivan Vučica
on
As discussed on Prosody MUC, I did, and the query is posted with LIMIT 1.
<iq to="aa@bb.net" id="something" type="get">
<pubsub xmlns='http://jabber.org/protocol/pubsub'>
<items node='urn:xmpp:microblog:0'/>
</pubsub>
</iq>
sql debug [select] SELECT "key", "type", "value", "when", "with" FROM "prosodyarchive" WHERE "host" = 'bb.net' AND "user" = 'aa' AND "store" = 'pep_urn:xmpp:microblog:0' ORDER BY "sort_id" DESC LIMIT 1;
Zash
on
I suspect that the reason is that it holds onto a reference to the node config from when the storage layer is initialized. That config object is then replaced when configuring. Earlier code updated the config object in-place.
Ivan Vučica
on
> That config object is then replaced when configuring
Do I understand correctly that, if I were to reconfigure the PEP node, it should adhere to the new value?
<iq type='get'
to='aa@bb.ccc'
id='config1'>
<pubsub xmlns='http://jabber.org/protocol/pubsub#owner'>
<configure node='urn:xmpp:microblog:0'/>
</pubsub>
</iq>
<iq id='config1' type='result' to='aa@bbb.cc/Gajim'>
<pubsub xmlns='http://jabber.org/protocol/pubsub#owner'>
<configure node='urn:xmpp:microblog:0'>
<x type='form' xmlns='jabber:x:data'>
<field type='hidden' var='FORM_TYPE'>
<value>http://jabber.org/protocol/pubsub#node_config</value>
</field>
<field type='text-single' label='Max # of items to persist' var='pubsub#max_items'>
<value>1</value>
</field>
<field type='boolean' label='Persist items to storage' var='pubsub#persist_items'>
<value>1</value>
</field>
</x>
</configure>
</pubsub>
</iq>
<iq type='set'
to='aa@bbb.cc'
id='config1'>
<pubsub xmlns='http://jabber.org/protocol/pubsub#owner'>
<configure node='urn:xmpp:microblog:0'>
<x xmlns='jabber:x:data' type='submit'>
<field var='FORM_TYPE' type='hidden'>
<value>http://jabber.org/protocol/pubsub#node_config</value>
</field>
<field var='pubsub#persist_items'><value>1</value></field>
<field var='pubsub#max_items'><value>50</value></field>
</x>
</configure>
</pubsub>
</iq>
<iq id='config1' type='result' to='ivan@vucica.net/GajimAtWork'/>
Upon retrying the 'get' of the config, the value is indeed updated. Upon fetching the items in the microblog, I still receive only one.
I should really get that automation tool you mentioned in MUC to work, and update the script you quoted (the 'advanced PEP' or something similar). Alternately, I'll write a bash script based on /usr/bin/expect (I already hacked something based on that for login availability monitoring).
Zash
on
This issue is actually in util.pubsub, our internal PubSub library, not mod_pep_plus. mod_pubsub would also be affected.
I've created a small test case so far that reproduces it with only util.pubsub. (Using Busted, the test framework we switched to in trunk.)
Changes
titlemod_pep_plus: Cannot fetch more than one message util.pubsub: Node config cached in storage abstraction and not updated
In mod_pep_plus, I've created multiple entries in urn:xmpp:microblog:0. By inspecting the PostgreSQL database, I can see that the items are there. However the following seems to fetch only one item. <iq type='get' to='my@bare.jid' id='items3'> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <items node='urn:xmpp:microblog:0'/> </pubsub> </iq> Same if I set max_items="5" (as an experiment). If I delete an item, the next one rears its head. If I fetch by a specific ID, I get the older items from the database, too. This is on Docker image based on Ubuntu Zesty. This is from Prosody trunk: 0de0018bdf91 As far as I can tell, mod_pep_plus is now only in trunk, so that's the version used.
What I failed to mention is that yes, I have increased max_items on the node and confirmed that its persist_items is true :-)
Did you check the SQL executed with mod_debug_sql? If you could try with internal storage too, that would be helpful to figure out what limits it to one item.
ChangesAs discussed on Prosody MUC, I did, and the query is posted with LIMIT 1. <iq to="aa@bb.net" id="something" type="get"> <pubsub xmlns='http://jabber.org/protocol/pubsub'> <items node='urn:xmpp:microblog:0'/> </pubsub> </iq> sql debug [select] SELECT "key", "type", "value", "when", "with" FROM "prosodyarchive" WHERE "host" = 'bb.net' AND "user" = 'aa' AND "store" = 'pep_urn:xmpp:microblog:0' ORDER BY "sort_id" DESC LIMIT 1;
I suspect that the reason is that it holds onto a reference to the node config from when the storage layer is initialized. That config object is then replaced when configuring. Earlier code updated the config object in-place.
> That config object is then replaced when configuring Do I understand correctly that, if I were to reconfigure the PEP node, it should adhere to the new value? <iq type='get' to='aa@bb.ccc' id='config1'> <pubsub xmlns='http://jabber.org/protocol/pubsub#owner'> <configure node='urn:xmpp:microblog:0'/> </pubsub> </iq> <iq id='config1' type='result' to='aa@bbb.cc/Gajim'> <pubsub xmlns='http://jabber.org/protocol/pubsub#owner'> <configure node='urn:xmpp:microblog:0'> <x type='form' xmlns='jabber:x:data'> <field type='hidden' var='FORM_TYPE'> <value>http://jabber.org/protocol/pubsub#node_config</value> </field> <field type='text-single' label='Max # of items to persist' var='pubsub#max_items'> <value>1</value> </field> <field type='boolean' label='Persist items to storage' var='pubsub#persist_items'> <value>1</value> </field> </x> </configure> </pubsub> </iq> <iq type='set' to='aa@bbb.cc' id='config1'> <pubsub xmlns='http://jabber.org/protocol/pubsub#owner'> <configure node='urn:xmpp:microblog:0'> <x xmlns='jabber:x:data' type='submit'> <field var='FORM_TYPE' type='hidden'> <value>http://jabber.org/protocol/pubsub#node_config</value> </field> <field var='pubsub#persist_items'><value>1</value></field> <field var='pubsub#max_items'><value>50</value></field> </x> </configure> </pubsub> </iq> <iq id='config1' type='result' to='ivan@vucica.net/GajimAtWork'/> Upon retrying the 'get' of the config, the value is indeed updated. Upon fetching the items in the microblog, I still receive only one. I should really get that automation tool you mentioned in MUC to work, and update the script you quoted (the 'advanced PEP' or something similar). Alternately, I'll write a bash script based on /usr/bin/expect (I already hacked something based on that for login availability monitoring).
This issue is actually in util.pubsub, our internal PubSub library, not mod_pep_plus. mod_pubsub would also be affected. I've created a small test case so far that reproduces it with only util.pubsub. (Using Busted, the test framework we switched to in trunk.)
Changesmod_pep_plus: Cannot fetch more than one messageutil.pubsub: Node config cached in storage abstraction and not updatedFixed in https://hg.prosody.im/trunk/rev/935e5ca43792 Part of the problem was what the title says not what my above comment says
Changes