undefined

points

[-]

This hits a sweet spot I think for conversations too. I've been playing (for quite a while) on trying to encapsulate long running conversations.

You have the overriding context, facts that don't change very often at all. The participants names, their backgrounds etc.

Then you have some very fine grained facts (what they ate for breakfast this morning) which might be useful right now, but are irrelevant outside of a general trend over the longer term.

When trying to reconstruct a conversation you really need to find the right balance without pulling in everything that has ever been discussed.

This definitely is worth further investigation.

by timwis3 hours ago|

parent|

[-]

Can you say more about how this applies to long-running conversations? I've been thinking about them as well, but can't write wrap my head around how this would be better than (or even different to) standard compaction.

by dominotw2 hours ago|

parent|

[-]

standard compactions doesnt really distinguish between long term vs short term ephemeral facts ?

by timwis2 hours ago|

parent|

[-]

Forgive me if I'm being naive, but can't you just tweak the compaction prompt to differentiate? Presumably that's what you would do in the separate prompt anyway, right?

by ewild5 hours ago|

parent|

prev|

[-]

This sounds like we are trying to add an LSTM into a transformer

by htrp4 hours ago|

parent|

[-]

Sepp would like a word

by storywatch3 hours ago|

prev|

[-]

Haven't read the full paper but thr local generation window is a little small, especially since image inputs are especially token heavy. Depending on where the local attention layer is located, it would be nicer if it's bigger e.g. 4096 words at least.

by MattRogish3 hours ago|

prev|

[-]

I do OCR of images, and that's exactly what I do. I take one big image and slice it into many smaller ones, and send those to the LLM. Perfect every time, unlike using the whole image which resulted in hot garbage.

by freefaler3 hours ago|

parent|

[-]

It works with relatively good scans, when there are bad/skewed scans and especially something with many label/value pairs, that aren't nicely tucked inside sentences, the more context you have, the more you can find the correct words and fix the errors.

There is a whole class of tricky documents. A decent (if you ignore the marketing bias) post about this problem can be found here:

https://getomni.ai/blog/ocr-benchmark

by ryanisnan2 hours ago|

parent|

prev|

[-]

How do you know where to slice an image? What if you slice an image mid-word?

by MattRogish1 hours ago|

parent|

[-]

I calculate* the appropriate overlap and the slicer overlaps a certain amount of the previous slice. There is some post-processing assembly required, but it's trivial.

[*] SWAG line height, trial and error to figure out the right amount of overlap given LLM error rates, etc.

by ryanisnan1 hours ago|

parent|

[-]

Interesting. Do you have a uniform data set? E.g. documents of a specific type that you know consistently have similar formats, or is this training something you need to do per-document?

by d6756 hours ago|

prev|

[-]

See, leetcode is useful. As I do this leetcode grind, I’ve been why techniques exist / how they’re used irl. Lots of interesting stuff there

by ai_fry_ur_brain5 hours ago|

parent|

[-]

Who said it wasnt useful, dont listen to those people.

by Xevion5 hours ago|

parent|

[-]

People who are applying to jobs and are tested with LeetCode problems to assess their skill level, despite the two not really being correlated or relevant for the position

by galbar5 hours ago|

parent|

[-]

As someone that gets very annoyed when having to do LeetCode in interviews...

Knowing algorithms, data structures and their memory and time complexities is very relevant for SWE. I've had teammates that didn't understand them and everything was fine until when it wasn't (scaling and performance issues).

Or, as I put it to a teammate: "Would you rather review the PR of someone that understands the difference between a set and a list or the PR of someone who doesn't?". This was after we interviewed a candidate with ~15 YoE, on paper, that didn't know the difference.

by elliottcarlson4 hours ago|

parent|

[-]

> Knowing algorithms, data structures and their memory and time complexities is very relevant for SWE

Agree with this; however knowing how to roll your own BFS/LRU/etc isn't -- in that case I'd rather review the PR of someone who understands how to leverage tested and known implementations than the PR of someone who decided to roll their own.

by ai_fry_ur_brain4 hours ago|

parent|

prev|

[-]

Who care's if the leetcode question doesn't relate to the job itself, it shows whether or not the person is willing to put in the work and gives you a glimpse into their ability to reason about hard problems.