Re: Measuring replay lag

On 14 March 2017 at 07:39, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com> wrote:
>
> On Mon, Mar 6, 2017 at 3:22 AM, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
>> On 5 March 2017 at 15:31, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>>> What we want from this patch is something that works for both, as much
>>> as that is possible.
>>
>> If it shows a sawtooth pattern for flush lag, that's good, because
>> it's truthful. We can only flush after we replay commit, therefore lag
>> is always going to be sawtooth, with tooth size approximating xact
>> size and the baseline lag trend representing any sustained increase or
>> decrease in lag over time.
>>
>> This would be extremely valuable to have.
>
> Thanks for your detailed explanation Craig. (I also had a chat with
> Craig about this off-list.) Based on your feedback, I've added
> support for reporting lag from logical replication, warts and all.
>
> Just a thought: perhaps logical replication could consider
> occasionally reporting a 'write' position based on decoded WAL written
> to reorder buffers (rather than just reporting the apply LSN as write
> LSN at commit time)? I think that would be interesting information in
> its own right, but would also provide more opportunities to
> interpolate the flush/apply sawtooth for large transactions.
>
> Please find a new version attached.

My summary is that with logical the values only change at commit time.
With a stream of small transactions there shouldn't be any noticeable
sawtooth.

Please put in a substantive comment, rather than just "See explanation
in XLogSendPhysical" cos that's clearly not enough. Please write docs
so I can commit this.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services