Brad Fitzpatrick ([info]bradfitz) wrote in [info]lj_dev,
@ 2002-03-03 23:12:00
Previous Entry  Add to memories!  Tell a Friend!  Next Entry
Entry tags:esn

ESN system

Update (2006-09-07): This post, while of historical interest, doesn't reflect the underlying implementation that finally happened in mid-2006. For actual data structures, see LJ/ESN.pm, LJ/Event.pm, LJ/Subscription.pm, etc...

This post is a quick introduction and overview of the forthcoming Event, Subscription, and Notification system, or ESN.

Basically, a ton of actions on the site will now fire events. An event type is technically 1 char for its type, 4 byte journal id, and 4 byte optional numeric argument. An actual event instance also has two other 4 byte fields, usually used for doer userid, and the new result id.

The API to fire an event is already in CVS, and talkpost_do.bml already uses it, though it does nothing yet, because....

Events only log to the database if somebody is subscribed to that event type. For each subscription, a user can choose their notification type. Initially only email and text message will be supported. Others in the future could be some web log of notifications or something.

Logging an event is O(1)[1] ... we log either 0 or 1 rows to the database, and this is done synchronously. Everything else is async: something will continually take an event off the front of the queue, find its subscribers, and send/pend notifications. (pend is future: for now we'll just send, but in the future we'll batch notification up such that they're not sent more than once every 5 minutes or something)

Types of notifications to be supported:
(J,journalid)+(ditemid) -- journal is updated (ditemid = which)
(R,journalid,ditemid)+(duserid,dtalkid) -- post is replied in
(T,journalid,dtalkid)+(duserid,dtalkid) -- thread is replied in
Other possible examples:
(P,journalid)+(picid) -- journalid has a new picture, and which
(B,journalid) -- new bio
(F,journalid)+(friendofid) -- friendofid added journalid as friend (only self and com. maintainers will be able to subscribe to this)

As far as paid/free ... paid users will be able to subscribe to anything. Free users will be able to subscribe to paid users' stuff, but with a cap on number of subscriptions.

What I need from somebody: the BML page to add/edit/delete your subscriptions. Anybody down?

[1] well, technically b-tree inserts aren't O(1), but you know what i mean... it's not O(subscribers). that's what's important.

(Post a new comment)


[info]halkeye
2002-03-03 11:21 pm UTC (link)
i'll attempt it after midterms are done (tuesday afternoon)
unless someone wants todo it earlier

(Reply to this)


[info]evan
2002-03-04 12:37 am UTC (link)
Might want to consider letting people block subscriptions. (I know it's silly 'cause the blockee can reload periodically, but I think a lot of people might get mad anyway.)

(Reply to this) (Thread)

[info]ntang
2002-03-04 05:11 am UTC (link)
How would the subscribee even know they were being "watched"?

I'm personally leaning towards the "if you don't want this person subscribing to this stuff, post it at a security level they can't see" idea.

(Reply to this) (Parent)(Thread)

[info]clith
2002-06-06 12:12 pm UTC (link)
Nit pick: a "subscribee" is usually called a "publisher"


:-)

(Reply to this) (Parent)(Thread)
Re:
[info]ntang
2002-06-06 12:34 pm UTC (link)
I have three things to say to you.

1.) Depends how you look at it. I was trying to distinguish between "publishers than know they're publishing, i.e., pushing data" and "people that are getting subscribed to, unwittingly. If someone peeks through your shades to watch you undress, it could be argued you aren't publishing that view to them. In the context of the discussion I think it made sense.

2.) It was also partially motivated by his use of "blockee".

3.) *phhhhbbbttttt*

Good day, sir!

(Reply to this) (Parent)
let all of us find out
[info]worldpeacedir1
2006-09-19 09:12 pm UTC (link)
let us see what is in return from all of this before we judge

(Reply to this) (Parent)
question.
[info]roy
2002-03-04 03:44 am UTC (link)
if someone were to subscribe to updates of my journal, would i be alerted in any way or have a way to find out?

(Reply to this) (Thread)
Re: question.
[info]bc
2002-03-04 04:42 am UTC (link)
Good question. I was going to ask that.
And I also like [info]evan's suggestion on blocking.
The two together would make most happy, I believe. (Notification and Blocking).

(Reply to this) (Parent)(Thread)
Re: question.
[info]roy
2002-03-04 05:48 am UTC (link)
i agree.

(Reply to this) (Parent)
Re: question.
[info]worldpeacedir1
2006-09-18 04:52 pm UTC (link)
I agree notification on blocking wud work better

(Reply to this) (Parent)

[info]ntang
2002-03-04 05:12 am UTC (link)
Initially only email and text message will be supported.

Gah, text messaging support. Now I really need to make sure everything's working. Speaking of which, I'm working on the next version, I'll send it to you probably this coming weekend, maybe sooner if I get the time.

(Reply to this)

Message Queue
[info]crackmonkey
2002-03-04 07:42 am UTC (link)
Have you guys considered using an open source Pub/Sub system instead of writing all this from scratch? You have the message definition there already.

I only know of OS Java MQs, but they do what you seem to be needing. Reliable delivery, queue/topic management, and even if they don't have the security API you're looking for, it's easy enough to conjure up a workaround.

Anyway, just a thought. I'd be willing to work on this for LJ, if there's interest. I have yet to actually contribute because my perl foo is very weak.

(Reply to this)


[info]xb95
2002-03-04 08:15 am UTC (link)
Oooh. I like. This should be client accessible. (Or if it's not, I'll make it so.) I'd like clients to be able to actually see what updates are happening instead of just seeing "There are new friends posts!" :)

(Reply to this) (Thread)

[info]mart
2002-03-04 05:13 pm UTC (link)

That'll be easier if clients ever start using a persistant connection protocol. Until then, polling for a boolean value is bad enough let alone polling for a list of events...

It'd be butt-ugly.

(Reply to this) (Parent)(Thread)

[info]chrisg
2002-03-07 04:39 pm UTC (link)
That would be really nifty.

Being able to have your client notify you on just certain events, etc just makes sense. Probably less trouble for the servers too? But I don't really know much about that part.

(Reply to this) (Parent)(Thread)
well let us both wait
[info]worldpeacedir1
2006-09-19 09:09 pm UTC (link)
let us both wait to see on the finishing touches of this point
uno it is never over untill the fat lady sings

(Reply to this) (Parent)

[info]deus_x
2002-03-20 09:38 pm UTC (link)
I would absolutely lurve to build an RSS feed out of selective event subscriptions to keep extra tabs on my Friends. :)

(Reply to this)

Missing event
[info]bostonsteamer
2002-04-03 03:22 pm UTC (link)
One event that was overlooked is polls. A poll creater should be able to get a notification when someone has taken his/her poll.

(Reply to this) (Thread)
Re: Missing event
[info]worldpeacedir1
2006-09-18 04:57 pm UTC (link)
I believe a poll should get that those who put on a good poll on OK stating on the president poll usa, Iraq ,Afghanistan, something informative

(Reply to this) (Parent)(Thread)
Re: Missing event
[info]bostonsteamer
2006-09-18 08:39 pm UTC (link)
wtf?

(Reply to this) (Parent)(Thread)
Re: Missing event
[info]worldpeacedir1
2006-09-19 09:02 pm UTC (link)
Kofi Anan painted a grim picture of the world

(Reply to this) (Parent)
status?
[info]lensovet
2004-03-29 06:12 pm UTC (link)
this post originates from a year ago. what's the status on this system? how soon will we see it implemented (i.e. become available to users)?
in addition, the restriction on free users is going to drive people mad. it seems too much like trying to coerce people into buying the service if they want their friends to easily be notified of new events in their journal.
...just a thought...

(Reply to this) (Thread)
oops
[info]lensovet
2004-03-29 06:13 pm UTC (link)
the thing is from two years ago! going online soon?

(Reply to this) (Parent)(Thread)
Re: oops
[info]duskwuff
2006-09-06 12:25 am UTC (link)
Online. :-)

(Reply to this) (Parent)

[info]raventhon
2004-05-09 03:09 pm UTC (link)
Would this notification apply to specific support queries as well?

(Reply to this)


[info]thrashmachine
2006-09-08 02:09 pm UTC (link)
kind of like the new facebook that everyone hates?

(Reply to this)


[info]suspect_device1
2006-09-08 08:49 pm UTC (link)
gay

(Reply to this)


[info]septembersky
2006-09-09 05:03 am UTC (link)
This is already a huge deal on facebook, people don't like it. I wouldn't recommend it. Everyone's very upset at my school over it.

(Reply to this) (Thread)
Why?
[info]wapogipo88
2006-09-09 06:34 pm UTC (link)
Why? They said it was non-stalkery.

(Reply to this) (Parent)(Thread)
Re: Why?
[info]septembersky
2006-09-09 06:49 pm UTC (link)
Because they think it's annoying and don't want to be notified every time one of their friends "friends" someone else or adds a new band. And also, just because they say it won't be stalkery doesn't mean it won't be.

(Reply to this) (Parent)(Thread)
Re: Why?
[info]wapogipo88
2006-09-09 09:08 pm UTC (link)
Aah, ok. That's what I thought.

(Reply to this) (Parent)
Re: Why?
[info]septembersky
2006-09-09 06:51 pm UTC (link)
Here's the post:

http://community.livejournal.com/emersoncollege/1374485.html

(Reply to this) (Parent)(Thread)
Re: Why?
[info]wapogipo88
2006-09-09 09:09 pm UTC (link)
Interesting!

(Reply to this) (Parent)

[info]major_kerina
2006-09-10 09:35 pm UTC (link)
Looks more like it just messed up Livejournal for all Firefox users than anything beneficial but I'll wait and see till it's all done but as a paid user I'm rather ticked. I was quite pleased with things as they were.

(Reply to this)


[info]vanillacocktail
2006-09-11 06:43 am UTC (link)
Have you actually tried it? Because it doesn't sound too bad.

(Reply to this)

hey
[info]worldpeacedir1
2006-09-19 09:06 pm UTC (link)
Those who are paying don't blame you of what is going on

(Reply to this)

update: 2006-09-07
[info]cyber_pilgrim
2006-09-30 03:01 pm UTC (link)
Now I thought I was getting the hang of computing, but after reading your update it's obvious to me I'm still really wet behind the ears. I never got the hang of dos, so your technical terms are all Dutch to me; even though it may not be dos.

I hooked up with your site as I was curious about the link to what was new. Considering the progress I've made in six months, it may make sense to me once the features become familiar. I only signed up two days ago, and blogging sure is different to negotiating around my Tripod web sites. I hope you don't mind the musings of a rookie.

(Reply to this)

Create an Account
Forgot your login or password?
Login w/ OpenID
English • Español • Deutsch • Русский…