distinguish POSSE posts vs non-POSSE mentions and handle accordingly #51
Description
this would be nice for catching when other people post a link to your post in a silo.
i did this for a while in mid 2012, before bridgy's re-release with webmentions. i stopped because the POSSEd posts showed up as comments on the original posts, and i kept that decision in the re-release because i didn't see enough people using rel-syndication links, which meant i couldn't prevent the same thing happening to them.
on the other hand, we've been thinking more about de-duping and similar issues recently, and @tantek proposed that this kind of noise might help motivate people to make their mention handling smarter. worth a thought.
Activity
snarfed commentedon Jan 31, 2014
concretely, these would only differ from current webmentions in that they wouldn't have an in-reply-to, since they truly are "mentions."
[-]send webmentions for original POSSE silo posts[/-][+]send webmentions for posts as well as responses[/+]snarfed commentedon Apr 14, 2014
two possible approaches for distinguishing the original author's POSSEd posts:
both are reasonable, and this would be a good feature. promoting to now.
snarfed commentedon Aug 26, 2014
lots of discussion about this on IRC today.
summary: when tweet links to a post, but isn't the official POSSE tweet of that post, responses are backfed and rendered as if they were responses to the original post. two examples. some people like this somewhat (e.g. @snarfed, @kevinmarks, maybe @kylewm); others don't (@aaronpk, @tantek).
it's hard to prevent this. @tantek correctly notes that we can use
rel=meto identify the original author, and only treat their tweets as POSSE candidates. that's a good step.however, the common case is that the original author later links to their post from a different (non-POSSE) tweet. we could use
u-syndicationand permashortcitations to distinguish that from the original POSSE tweet, but both of those have low adoption rates among bridgy users, so we'd end up muzzling the majority of responses, which i don't want to do.@kevinmarks suggests that we use time as a heuristic. if the author links to their post over 24h after it's originally posted, don't consider that a POSSE. definitely a good idea!
(i'd re-emphasize that this is all tradeoffs. given real world usage, i don't see a single best answer so far, and leaving the current behavior is on the table. good to hash through options though!)
[-]send webmentions for posts as well as responses[/-][+]send webmentions for (non-POSSE) posts as well as responses[/+]snarfed commentedon Jan 8, 2015
current proposal from @tantek in IRC today: only consider a link to be the original copy if it's on a domain in the user's silo profile. sounds ok to me, we could consider implementing it.
tantek commentedon Jan 8, 2015
As written up: http://indiewebcamp.com/original-post-discovery#POSSE_copy_domain_approximation
karadaisy commentedon Apr 13, 2015
✋ In case it's useful, here's an example where Bridgy is being overly aggressive in assuming a tweet is the POSSE copy of an original.
here's the original: https://adactio.com/journal/8710
here's a tweet from someone else (another bridgy user) linking to the original: https://twitter.com/jgarber/status/587245857034133504
and then a bunch of RT's of that tweet are backfed to the original as if they are RTs of the original. e.g., https://brid-gy.appspot.com/repost/twitter/jgarber/587245857034133504/587680705938907136
snarfed commentedon Apr 13, 2015
thanks @kylewm!
one way to mitigate: when the post's domain isn't one of the tweet author's domains, demote to u-mention.
58 remaining items
snarfed commentedon Sep 15, 2015
i've updated the discussion of these OPD heuristics in https://indiewebcamp.com/original-post-discovery#Brainstorming . tldr: there are four, and we've hit real world counterexamples for all of them in bridgy, so none are ideal.
karadaisy commentedon Sep 15, 2015
few random thoughts...
Another possible heuristic: have we already seen a POSSE for this post on this service? if so, it's more likely that subsequent links are mentions. It's not that strong of a criteria because many people will tweet links to the same piece throughout the day (e.g. Dave Winer), and of course tweets are deleted and reposted as edits.
It's much more costly to incorrectly identify a POSSE copy as a mention, i.e. no backfeed for that post. So the threshold for qualifying as a POSSE copy should probably be way lower, maybe matching some subset of the criteria, like off the top of my head:
It's very difficult to correctly categorize the "Kevin tweets a link to his post within 24h" case without throwing out a lot of legitimate POSSEs. In the specific case on the wiki, we could say it looks like he is tweeting at someone but the original isn't in-reply-to anything...wonder if that applies more generally to self-mentions.
snarfed commentedon Sep 15, 2015
thanks @kylewm! interesting idea to record inferred POSSE links and check them later. kind of an extension of the way we already store syndication links. and you're right, the standard way to handle a complicated inference like this based on heuristics is to combine them with weights into a score... and that in this case, false negatives hurt much more than false positives. (I've always described bridgy as deliberately "promiscuous." :P)
I'm already second guessing all this added complexity, though, and it looks like the domain check is comfortably the strongest so far, so I'm kind of leaning toward just that. meh.
karadaisy commentedon Sep 15, 2015
I would support that too. Fight that sunk cost fallacy!
snarfed commentedon Sep 23, 2015
@tinokremer sorry for the trouble! you're right, it probably is due to this. current status: trying to track down the memory leak in #456 (comment), which is blocking further fixes here. wish me luck!
PPD: for redirects, use final URL for domain check, and attach pre-re…
add include_redirect_sources kwarg to Source.original_post_discovery()
on redirects, only include final URLs in webmention targets, not init…
document new domain check for POSSE vs mention logic
snarfed commentedon Sep 26, 2015
tentatively closing. this has been running in prod and stable for a few days. I'm sure there are more bugs left to fix, but we can open new issues for them.
relax OPD domain req't to allow subdomain match. for snarfed/bridgy#51
don't backfeed responses to link posts unless they're by the user