Loading Profile...
POSTing notifications to a URL
I'd really like to be able to POST an RSS update to any URL.
My specific reason is that I'm trying to get emails to be POSTed to a URL and I've found a service that converts emails into RSS feed items.
In general though, with this feature, Notify.me could be used to keep the flow of information going through different web services.
More about this here:
http://jaybyjayfresh.com/2009/03/26/e...
My specific reason is that I'm trying to get emails to be POSTed to a URL and I've found a service that converts emails into RSS feed items.
In general though, with this feature, Notify.me could be used to keep the flow of information going through different web services.
More about this here:
http://jaybyjayfresh.com/2009/03/26/e...
I’m optimistic
3
people like this idea
I like this idea!
Tell me when this idea gets some attention.
The more people who like this idea, the more it gets noticed.
The more people who like this idea, the more it gets noticed.
The company has this in progress.
The best point from the company
-
We're also working on our REST API for posting notifications into notify.me. The current draft of the documentation can be found here:
http://wiki.notify.me/User:Arne/Accou...
We're still tweaking the API as we develop it, but most of the "in progress" items are fairly nailed down in terms of implementation
I’m happy
The company thinks
this is one of the best points
Create a customer community for your own organization
Plans starting at $19/month
-
Inappropriate?You can send emails directly to your notify.me timeline by using the email address user@notify.me
Does this help or have I missed the point. We consider the email data private just like the subscribed feeds. We are planning to add a public/private source field soonish. -
Inappropriate?We're also working on our REST API for posting notifications into notify.me. The current draft of the documentation can be found here:
http://wiki.notify.me/User:Arne/Accou...
We're still tweaking the API as we develop it, but most of the "in progress" items are fairly nailed down in terms of implementation
I’m happy
The company thinks
this is one of the best points
-
Inappropriate?I want this to send the notification to a URL I specity... ala "webhooks" or smtp2web
I’m confident
-
Inappropriate?I would like to support this idea. It is not just about generating some RSS from a feed, but it is about really being able to do something with the feeds from notify.me. Notify.me is a great service, but it would be truely awsome if you just added a URL destination a la Jeff Linday's Webhooks (see here too)
I’m confident
-
Inappropriate?I'm currently working on the webhooks daemon for notify.me and thinking about the POST format for the initial release. I'm a RESTful guy, so my personal preference would be to simply POST an XML request body. However, most people are used to form data when dealing with a POST, so I am leaning towards defaulting to application/x-www-form-urlencoded key/value pairs, and offer XML later. Any preferences here?
I’m excited
-
Inappropriate?Arne, for me it would be best to have a nice RESTful deamon first. POSTing XML would cover far more possibilities for programming the results -- I would think. Passing feeds back as key/value pairs seems very limiting compared to XML, but that is just me.
I’m thankful
-
Inappropriate?@Arne
I'm not sure why you'd want to explicitly "POST an XML body"... What I think you'd want it to post whatever the service gave you, i.e. I don't think you should be a translation later or add any type of meta info except perhaps in the URL itself.
Not sure if that made sense but as a principle I'd say "leave the message alone" and then you're not forced to try and reinvent any wheels at all, nor do you end up with something nasty like an XML encapsulated JSON message.
My 2nd rule of thumb is JSON for everything over XML, since we are talking JSON here but I realize that's like saying VI rulz ;)
Any meta information you really wanted to pass on should be in the request parms, e.g. &from=twitter@twitter.com
I’m thankful
-
Inappropriate?The initial version will not be able to act as a delivery agent for arbitrary message bodies, so we won't be sending the full rss items we're parsing, but basically just the data we currently deliver, which really comes down to from, subject and ref-uri for now. So the options really are just serialize that data into xml or key/value pairs. Clearly the key/value pair format is very limited but also the most common.
Once we get a bit further along, we will probably also deliver message bodies, which will be allow for arbitrary data payload (although we will likely still cap the message sizes). At that point XML will be the main format, although we'll also over JSON encodings (which may loose some of the meta-data).
I do not favor meta data in request parameters on post. If there is meta information that does not belong into the message format, it will be represented by message headers.
Either way, in all cases except for the application/x-www-form-urlencoded format, i will be delivering a message envelope of a known schema, even if that schema also includes a raw body field.
I’m excited
1 Comment Add a comment
This is one of the best points
-
I do think spamming request parms is ugly but I think the fact that you "will not be able to act as a delivery agent for arbitrary message bodies" is a concern.
I think starting with "we support the generic anything case" seems a much better approach because it scales the fastest. Me having to rely on you guys supporting email, rss, atom, webhooks, etc... all on your own time (and priorities) is a huge issue. As an example, I have a twitterbot ready to go but the fact that I can't get the SMTP messages translated (reliably) into POST data has stalled me for weeks. -
Inappropriate?I must stress that I am not a systems person, nor much of a programmer either, but I am interested in how these services get used. It seems to me that what is needed is not a message delivery service here, but a notification service (seems obvious, but may not be). What I mean is that there may be some very good user reasons for only posting from, subject and uri, and not the full message body. Not only because there must be some very real capacity issues for notify.me, but, also, there are very good sociological reasons for letting the sender and receiver decide what to do with the uri.
The power of webhooks, as I understand it from Jeff Lindsay, is that it takes on-board a RESTful world, and sees its role as pushing the bare minimum of information from a provider to a user. However, what is at the end of that uri, and what the user can do with it, may be vast. However, what they both do with that is up to them. I think that there are good non-technical reasons why this light touch is more powerful than a metadata heavy message passing one.
Hope this all makes some sort of sense.
I’m excited
1 Comment Add a comment
This is one of the best points
-
Robin, I think the point you make about "notification" vs "message delivery" is an interesting one. I tend to think that if you're going to notify me that there's a new message(s) waiting you might as well just deliver the message itself in most cases but if it's just a ping to wake me up to go poll RSS that would work too.
-
Inappropriate?Personally I would find form-urlencoded key/value pairs most useful, as then you can notify applications as if you've just POSTed from their own form. I'm thinking Yahoo! Pipes here.
I’m RESTful
1 Comment Add a comment
This is one of the best points
-
My concern with key/value pairs is that, for example, the python email library supports concerting a string into an email object. In my experience that's how most libraries are used to inputs, e.g. BeautifulSoup and simplejson parse strings.
If you hand me key/value pairs I'm going to have to shoehorn that input to make it acceptable for just about every library out there. -
Inappropriate?Actually, after the last few comments from Arne Claassen, which clarified that this will not be a messaging service, and after some more thought, POSTed key/value pairs would be fine.
I’m also RESTful, now
1 Comment Add a comment
This is one of the best points
-
I think I agree with Robin at this point. If you can't commit to delivering me a string representation of the message, then don't deliver me much at all (just the service URL to go check).
-
Inappropriate?Robin,
If you are looking at our WebHooks from a RESTful perspective, the question is what is the resource the hook is referring to. We consider the resource to be the notification stored in our system, which itself may refer to another resource that is web accessible (such as an RSS item) or it may refer to a resource that exists only in our system. The idea of the hook is that it notifies you of the resource being created or changed and provides you a canonical Uri for the resource.
Now this presumes that we can give you a canonical Uri to our resource, the API for which may lag a little behind the webhook rollout. But for notifications based on RSS items, we will be passing through the Uri to the item itself.
Over time the hook may include more or less of the actual resource, as we see how people are using them, but in the end the hook is just a notification about a resource.
Sounds like i'll go with application/x-www-form-urlencoded until we can expose more sophisticated configuration options for webhooks.
I’m excited
7 Comments Add a comment
This is one of the best points
-
My view on what webhooks say is that there is no "question" about what it's referring to because it's be definition open... i.e. it doesn't specify what's being transported or to where, other then it's via an HTTP verb.
I think your "wiring" gives me a canonical URL to your servers which I will have to lookup and dereference to get to the actual data. Not necessarily bad if you could provide meta info on that, e.g. historical information, i.e. change control on a REST resource, that is assuming I wanted it for my context.
Sounds like for now you're just going to; (a) alert my URL handler then (b) proxy the URL request I make to the actual resource
I'm not sure I see the value in (b) but that might just be my desire to optimize a step (and a point of failure) out of the equation. -
@thecapacity You might find this useful: http://www.smtp2web.com/
-
@thecapacity From your posts, it seems that what you really desire is a transparent transport that takes any input and re-delivers it as an Http post. While I agree that that is a useful service, it isn't quite aligned with our model for notify.me, i.e. we're more a notification than a re-routing service.
I agree with your concern in regards to (b). It is true that we currently use our own url shortener when we send out notification. We do this for two reasons, (1) to make sure we have a shortened url that works in email, IM, etc. and (2) to get analtyics on the usage of that notification. Since webhooks is more of a plumbing than a direct consumer delivery using the proxied url is less significant. I will have to talk to the rest of the team regarding losing the analytics provided in favor of passing through the true canonical url instead of the proxied one, since the proxy is really a violation of REST anyhow. -
@Jonathan thanks for the link I've actually been using smtp2web for a while not but it's really not a strong solution for production (that's not a criticism just an observation).
@Arne - Absolutely right that the transport munging is one critical aspect. I wouldn't know your business as well as you, but as I mentioned in an earlier post it seems splitting hairs to say "I'll notify you a message exists in this queue / transport type" but not just go ahead and deliver some digest (full or partial) of that message, i.e. the message itself can serve as notification.
I wouldn't think that you track the URL shorted version would be the best way to get a usage breakdown, i.e. my program may not follow that URL most / all of the time. I'd expect the only "certain metric" you could rely on is the act of delivering to my endpoint.
Webhooks is certainly "plumbing" but I wouldn't overlook it anymore then I'd say RSS, Email or even twitter & facebook are plumbing. I'm less sure I understand the nuances behind your statement that it's less "a direct consumer delivery" but I think that's what I see webhooks and RSS (in theory though it's of course reversed in practice) as.
Anyway, don't mean to come off as a contrarian but it's definitely a need I have, i.e. to convert from polling to something more responsive. I know there's Gnop (sp?) but can't seem to find it and having checked out Switchhub too I can tell you it's just a service vacant of any providers. -
I think that @thecapacity has hit the nail on the head here. Obviously it is not for me to tell you how to make notify.me work. You know that better than any of us. However, like @thecapacity I too need something like webhooks that not only gets me away from polling, but also allows my web services to post to other web services when things change. This may sound ridiculously simple, but, as Jeff Lindsay says, Webhooks are so simple you'll think they are stupid (http://blog.webhooks.org/2009/04/14/s...). The point is that I don't think they are -- not in the least.
Of course it may not be for notify.me, which I too think is a very nice service, but would it take that much to add such to notify.me? -
@thecapacity Tracking the shortened url isn't the best way, it is just one additional way. There is value in knowning that we deliverd the notification to someone and that their service accepted the post. There is additional value in knowing what percentage of those deliveries resulted in traffic back to the originating site. It's not an either/or.
@robin I do believe that what we will offer will get you away from polling, since we notify you of an event and where to get the details. However, we are not going to deliver the full content represented by the original resource, but leave that up to you to fetch given the Uri we sent in the notification. Please let me know if this leaves something out.
For us it's not a quesiton of how simple the delivery of the webhook is. It's a capacity issue of parsing, queueing and relaying all the items we process, which I can tell you is a lot :) Since a very large percentage of these notifications do not result in click-backs to the original source (i.e. people decide they found out enough from the subject or just knowing that something changed), delivering the full content that is likely ignored by the destination is not a worthwhile capacity proposition for us at this time. -
@Robin & @Arne
Both of you make good points, although I'm still in the camp w/ Robin (i.e. I'd like the data too). :)
I think Arne's point about being able to track the ratio of ping's to pongs (sorry that's how I envision them) is actually useful and you're right that it doesn't have to be exclusively one or the other.
I can only imagine the capacity issues you have to account for, but I do feel inclined to point out that with with the "webhooks" approach there's very little of the parsing that has to occur. I'd think it should just be a hash lookup, though of course the task of queuing and relaying is still there.
Anyway, I think I must apologize for trying to reply to each in return and creating the spaghetti approach to a discussion ;) -
Inappropriate?@Arne Claassen, Sorry I was out of the loop, but I have been in meetings. As I understand it, thecapacity is right here. Webhooks do not make a decision about what is passed and how it is dealt with. However, webhooks do have to know the source uri and the destination uri. Webhooks also need to take something from the source uri and pass it to the destination uri when some state changes. Jeff Lindsay's vision of WebHooks sees this as all one operation, but I am not sure he has yet thought about how this should be done in aggregation. This is where notify.me could come in.
So, I think that @thecapacity is right that, as an agregator, notify.me is very useful -- his (a) above, but I too see little point in (b). As long as I know that there is a URL I can use, and how to use it, the notification can trigger that use? Am I right here?1 Comment Add a comment
This is one of the best points
-
@Robin, thanks that's a much clearer explanation then anything I could have come up with!
-
Inappropriate?Much as REST has been misused from what Roy Fielding defined as, I guess Jeff Lindsay's WebHooks is destined for the same mutation, so I probably should stop calling what i'm building webhooks so no undue assumptions are made. This facility I am creating is another destination for notifications in the notify.me system, which happen use HTTP POST to deliver the notification. That's all they are.
So if what we deliver is sufficient (which is basically the same data you'd get via IM from us), then you're done. If however, you need the original material, our callback to your handler is merely a trigger for you to fetch the data. Whether or not the uri we provide in the notification is the canonical uri to the resource or a proxy uri that returns a 301 to the real Uri, is something that I don't know yet.
I’m excited
2 Comments Add a comment
This is one of the best points
-
I just wanted comment on this to clarify how I think I'd end up using this service;
1) I'd setup notify.me to alert me when there's a message, say on an RSS or email queue or whatever your support.
2) When I got pinged via notify.me I'd say "thanks for the alert" and then I'd simply go check what queue / service I'd registered to send to my URL
Now that I better understand what you're trying to do I think that probably sounds "aggressive" or "anti-competitive", i.e. I'm exploiting your service and not "giving back".
I really don't mean it to sound like that (I'm a big fan of supporting people to make possible the services they provide!!) but I thought I'd try to explain my "interpretation" in case it helped better present my perspective. -
@thecapacity I have no qualms about you using our service in such a way. Clearly we're offering that capability and if we didn't like that you could use it that way, it'd be our own fault. The caveat here is that the Uri you get is likely going to be one that hits our service first and then 301s back to the real Uri, because us doing this for free means we need to at least gather analytical data about statistical usage.
-
Inappropriate?Just to clarify, because there's been a lot of back and forth on the definitions... We plan to provide WebHooks that deliver via HTTP what overwise would be delivered via Xmpp, email or SMS. I.e. it's a small envelope information notifying being posted, including a uri to the information. That Uri may be one that simply 301s because it proxies through us for analytical purposes.
Does that serve your need, or is something vital missing that you cannot synthesize from the Uri we give you?
I’m german
1 Comment Add a comment
This is one of the best points
-
It sounds reasonable, I just wanted to try to better illustrate my "thought process" and why I think there would be more info if you catch the initial "ping" rather then the return "pong" :D
I really want the data to come across in addition to the ping so I can shave off a 2nd step of gathering it. But I can understand the size constraints... i.e. your comments to Robin below.
As a thought of XML vs. JSON, maybe that could be a configurable parm when I set up the endpoint? i.e an option that says "deliver this to me as <>" -
Inappropriate?@Arne, thanks for this clarification because I'm getting a bit confused as well. That almost certainly would do it, thanks. When you get it finished, let me know so I can play with it.
One point, though. I had a word or two with Jeff Lindsay who said, if I can paraphrase him correctly, that if you are going so far as to pass the uri and some other metadata, why not just pass the xml data? Obviously, there would have to be some limit on size, but if it is a small amount of info, why not just pass the data? Just passing this on.
Many thanks
I’m confident
1 Comment Add a comment
This is one of the best points
-
Because of the volume of messages that go through our system right now, and none of our other other endpoint supporting the full data, we don't have access to the original data when we call the webhook.
Passing along the original data up to a certain size limit, or even letting subscribers elect whether they want to receive the data are options we are exploring, but it won't make it into V1.
EMPLOYEE
EMPLOYEE
