no more hslv format ?

[chris chery]

hi
whatever happened on or arounf friday 13 th april when I downloaded hamlet
in hlsv 1200*720 25fpsand now when all I get is the Hvfxsd1format 960*540
25dfps
Trying to redownload hamlet i do get the hlsv size but no such luck with
any other progammes
i am outside the uk
regards
cc
----- Original Message -----
From: "Paul Thornett" <***@gmail.com>
To: "RS" <***@zoho.com>
Cc: <***@lists.infradead.org>
Sent: Wednesday, April 18, 2018 1:26 PM
Subject: Re: Cannot play downloads from get_iplayer!


> That's interesting and odd. With the --info option I don't see the
> hlsxx options nor the hvf... one, but I can certainly download the
> hlsxx one!
> Regards,
>
> Paul Thornett
>
>
> On 18 April 2018 at 20:41, RS <***@zoho.com> wrote:
>> On 18/04/18 11:05, Paul Thornett wrote:
>>>
>>> As a matter of interest, how do I check if hls is available for a
>>> particular file? I thought --info would show this, but using this
>>> option on "Ordeal by Innocence Episode 1" does not reveal the
>>> availability of the hlshd stream, even though it is there.
>>
>>
>> This is what --info gives me for that episode.
>> modes: editorial:
>> dvfhd1,dvfhd2,dvfsd1,dvfsd2,dvfxsd1,dvfxsd2,dvfxhigh1,dvfxhigh2,
>> dvflow1,dvflow2,hlshd1,hlsvhigh1,hvfhd1,hvfhd2,hvfhd3,hvfsd1,hvfsd2,
>> hvfsd3,hvfxsd1,hvfxsd2,hvfxsd3,hvfhigh1,hvfhigh2,hvfhigh3,hvfxhigh1,
>> hvfxhigh2,hvfxhigh3,hvfstd1,hvfstd2,hvfstd3,hvflow1,hvflow2,hvflow3,
>> subtitles1,subtitles2,subtitles3
>>
>> [line breaks inserted]
>>
>> I have just noticed that the only HLS modes are hlshd1 and hlsvhigh1. For
>> the other two episodes there were only DVF and HVF modes.
>>
>>
>> Best wishes
>> Richard
>>
>>
>> _______________________________________________
>> get_iplayer mailing list
>> ***@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/get_iplayer
>
> _______________________________________________
> get_iplayer mailing list
> ***@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

[chris chery]

hi i m in france
I just try to download hamlet again to see if i could get the hlshd1 format
,which I did get but all other download attempts gave me hvfxsd1
----- Original Message -----
From: Paul Thornett
To: chris chery
Sent: Wednesday, April 18, 2018 2:44 PM
Subject: Re: no more hslv format ?


Just because I'm interested, where are you located (I'm in Australia)? And
how come you need to download a 2nd time?

I downloaded Hamlet in hlshd format. It's 3.2Gb, and I'm happy to share it
with you if you like. I could also reduce its size using any parameters you
like.


-------------
Regards,
Paul Thornett



On Wed., 18 Apr. 2018, 21:55 chris chery, <***@free.fr> wrote:

hi
whatever happened on or arounf friday 13 th april when I downloaded hamlet
in hlsv 1200*720 25fpsand now when all I get is the Hvfxsd1format 960*540
25dfps
Trying to redownload hamlet i do get the hlsv size but no such luck with
any other progammes
i am outside the uk
regards
cc
----- Original Message -----
From: "Paul Thornett" <***@gmail.com>
To: "RS" <***@zoho.com>
Cc: <***@lists.infradead.org>
Sent: Wednesday, April 18, 2018 1:26 PM
Subject: Re: Cannot play downloads from get_iplayer!


> That's interesting and odd. With the --info option I don't see the
> hlsxx options nor the hvf... one, but I can certainly download the
> hlsxx one!
> Regards,
>
> Paul Thornett
>
>
> On 18 April 2018 at 20:41, RS <***@zoho.com> wrote:
>> On 18/04/18 11:05, Paul Thornett wrote:
>>>
>>> As a matter of interest, how do I check if hls is available for a
>>> particular file? I thought --info would show this, but using this
>>> option on "Ordeal by Innocence Episode 1" does not reveal the
>>> availability of the hlshd stream, even though it is there.
>>
>>
>> This is what --info gives me for that episode.
>> modes: editorial:
>> dvfhd1,dvfhd2,dvfsd1,dvfsd2,dvfxsd1,dvfxsd2,dvfxhigh1,dvfxhigh2,
>> dvflow1,dvflow2,hlshd1,hlsvhigh1,hvfhd1,hvfhd2,hvfhd3,hvfsd1,hvfsd2,
>> hvfsd3,hvfxsd1,hvfxsd2,hvfxsd3,hvfhigh1,hvfhigh2,hvfhigh3,hvfxhigh1,
>> hvfxhigh2,hvfxhigh3,hvfstd1,hvfstd2,hvfstd3,hvflow1,hvflow2,hvflow3,
>> subtitles1,subtitles2,subtitles3
>>
>> [line breaks inserted]
>>
>> I have just noticed that the only HLS modes are hlshd1 and hlsvhigh1. For
>> the other two episodes there were only DVF and HVF modes.
>>
>>
>> Best wishes
>> Richard
>>
>>
>> _______________________________________________
>> get_iplayer mailing list
>> ***@lists.infradead.org
>> http://lists.infradead.org/mailman/listinfo/get_iplayer
>
> _______________________________________________
> get_iplayer mailing list
> ***@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

[Jim web]

I've been discussing the 'loss' of the 1280x720 25fps version with someone
at the BBC.

IIUC this stemmed from 'Red Bee' days of yore, and until recently people at
the BBC had thought they had stopped it long ago. Someone apparently
noticed recently that it was still available. And then actually disabled
it. Hence the mysterious recent ending of its availablity.

I have made the point that the 1280x720 25fps version is useful for people
with a content 'cap' problem, etc, so will be missed because the 50fps
version means somewhat bigger files and/or stream rates. But I doubt this
will cause a rethink.

That leaves me currently in a dither between fetching the 1280x720 50fps
and running it though a frame rare conversion or giving up and making do
with the 960x540 version(s). It has also set me wondering about arranging
for gip to fetch to ram storage and then convert that into a file on my
main disc. (The machine I use does have 8GB of ram, mostly free when I am
fetching things in the morning.) Anyone have experience of doing this, file
by file?

What I have in mind is a process of

1) Use gip to get a 50fps file saved to ram.

2) Get ffmpeg to convert that into a 25fps version on main disc.

3) Delete the file in ram

4) Loop back to (1) for the next file on the list.

I can avoid a 'cap' as I do all the fetching before the 9am start of our
'metered' period. The above should reduce wear and tear on my main disc (an
SSD). But I don't know how feasible it might be. Comments?...

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[iz]

On 30 April 2018 at 13:56, Jim web <***@audiomisc.co.uk> wrote:
>
> 1) Use gip to get a 50fps file saved to ram.

You'll want to use the --raw flag with GiP to avoid remuxing to MP4
and tagging, both of which create a file copy, which would halve the
max size of programmes you could download to ram disk.

[Jim web]

In article
<CAF_ssT8Eyw7GuX105X037kTpoNxCQjXjmunzAFN-***@mail.gmail.com>, iz
<***@gmail.com> wrote:
> On 30 April 2018 at 13:56, Jim web <***@audiomisc.co.uk> wrote:
> >
> > 1) Use gip to get a 50fps file saved to ram.

> You'll want to use the --raw flag with GiP to avoid remuxing to MP4 and
> tagging, both of which create a file copy, which would halve the max
> size of programmes you could download to ram disk.

Yes. noted OK. :-) Following on from that, though...

How do I find out the command gip sends to ffmpeg to do its default
conversion as things stand? IIRC it has to recontain the content and tweak
the aac format in some way. So I'd need to be able to replilicate what is
needed, plus the spec to tell ffmpeg to fo 50 -> 25fps.

Basically, I'm looking for the quickest and most 'damage free' way to get
the frame rate conversion but still leave the audio and level of picture
detail OK.

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[iz]

On 30 April 2018 at 15:22, Jim web <***@audiomisc.co.uk> wrote:
> conversion as things stand? IIRC it has to recontain the content and tweak
> the aac format in some way. So I'd need to be able to replilicate what is

GiP uses ffmpeg's aac_adtstoasc bitstream filter to remove ADTS
framing when copying the audio stream from .ts files, but more recent
versions of ffmpeg do that automatically for MP4 output format. If you
re-encode the audio as well as the video (probably not necessary),
that won't matter.

[Ralph Corderoy]

Hi Jim,

> How do I find out the command gip sends to ffmpeg to do its default
> conversion as things stand?

Try `--verbose'; that should print external commands that are run.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Peter S Kirk]

Jim,

What ISP and plan is that?

On 30 Apr 2018 at 13:56, Jim web Jim web <***@audiomisc.co.uk> wrote:

> I can avoid a 'cap' as I do all the fetching before the 9am start of our
> 'metered' period.

[Jim web]

My ISP is Orpheus internet. They aren't cheap, but they are excellent in
terms of service, etc. [1]

I think my actual connection is a repackage of a BT/OR one that has a cap
at a rolling 50GB/30days. But only the time betweem 9am and midnight is
metered. So I can 'fill my boots' outwith that time if I wish. I then
typically get around 50-60 meg download rates since we got FTTC.

Jim

[1] They helpfully support a range of OS's including my actual favourite,
RISC OS. To give an idea of the service, the owner will give clients his
mobile number in case they need help out-of-hours. I once did at the
weekend, phoned him, and he was out sailing. He said he'd fix the problem,
and it was fixed a short time later.

In article <***@peter.kirk.isauk.biz>, Peter S Kirk
<***@isauk.biz> wrote:
> Jim,

> What ISP and plan is that?

> On 30 Apr 2018 at 13:56, Jim web Jim web <***@audiomisc.co.uk> wrote:

> > I can avoid a 'cap' as I do all the fetching before the 9am start of
> > our 'metered' period.



> _______________________________________________ get_iplayer mailing list
> ***@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[Ralph Corderoy]

Hi Jim,

> I've been discussing the 'loss' of the 1280x720 25fps version with
> someone at the BBC.

I miss those 1 GiB ~= 1 hour ones too. They were `just right'.

> It has also set me wondering about arranging for gip to fetch to ram
> storage and then convert that into a file on my main disc.

Is this Linux? get_iplayer here uses the current working directory for
all its large intermediate files. If that was a `tmpfs' filesystem then
they would all sit in RAM, unless you've swap space and pressure caused
pages from the filesystem to be swapped out.

You could use get_iplayer's --command option to run a command to move
each final file off tmpfs as the download is finished. Its --output
affects all the intermediate files too, AIUI.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Alan Milewczyk]

On 01/05/2018 12:31, Ralph Corderoy wrote:
> Hi Jim,
>
>> I've been discussing the 'loss' of the 1280x720 25fps version with
>> someone at the BBC.
> I miss those 1 GiB ~= 1 hour ones too. They were `just right'.
Me too! :-(
>
>> It has also set me wondering about arranging for gip to fetch to ram
>> storage and then convert that into a file on my main disc.
> Is this Linux? get_iplayer here uses the current working directory for
> all its large intermediate files. If that was a `tmpfs' filesystem then
> they would all sit in RAM, unless you've swap space and pressure caused
> pages from the filesystem to be swapped out.

I'm using Windows 7 not Linux but I wouldn't have a clue how to do that.
> You could use get_iplayer's --command option to run a command to move
> each final file off tmpfs as the download is finished. Its --output
> affects all the intermediate files too, AIUI.
>
I've run the suggested command line instruction posted earlier and I'm
afraid it's a very time consuming process - assuming nothing else is
going on on my PC I get a conversion speed of approx 1.5x/1.6x (down to
maybe 1.2x if other processes are active). Maybe some of that could be
due to the relative slowness of hard drives. It would be good to know
how to modify the code to convert the programme files on the fly.


Regards


Alan


---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

[Jim web]

In article <***@orac.inputplus.co.uk>, Ralph
Corderoy <***@inputplus.co.uk> wrote:
> Hi Jim,

> > I've been discussing the 'loss' of the 1280x720 25fps version with
> > someone at the BBC.

> I miss those 1 GiB ~= 1 hour ones too. They were `just right'.

Yes. It seems bonkers to me for the BBC to end them. But I've not had any
reaction to pointing out that they are useful, etc. So I assume they won't
be re-instated. :-/

> > It has also set me wondering about arranging for gip to fetch to ram
> > storage and then convert that into a file on my main disc.

> Is this Linux? get_iplayer here uses the current working directory for
> all its large intermediate files. If that was a `tmpfs' filesystem then
> they would all sit in RAM, unless you've swap space and pressure caused
> pages from the filesystem to be swapped out.

> You could use get_iplayer's --command option to run a command to move
> each final file off tmpfs as the download is finished. Its --output
> affects all the intermediate files too, AIUI.

Yes, Linux. The challenge for me is to work out how to get the fetched file
to go onto the tmpfs, then have that processed - ideally having gip give
ffmpeg relevant details - into a 25fps file on my main disc. i.e. I don't
know enough to specify the commands and either get gip to do this, or add
in the relevant ffmpeg call.

Am I right that by using the --command and also --raw I can then run ffmpeg
to do the conversion and put the result where I want? That sounds about
right for what I have in mind.

I'll look at what the --verbose gives me as per your other email. Someone
elsewhere has said they can get handbrake to do 50 -> 25 fps fairly well.
But that means running that as well unless I can work out what it is doing
on a command level. And IIUC handrake is using ffmpeg anyway under the
hood...

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[Steven Carr]

On 1 May 2018 at 14:05, Jim web <***@audiomisc.co.uk> wrote:
> I'll look at what the --verbose gives me as per your other email. Someone
> elsewhere has said they can get handbrake to do 50 -> 25 fps fairly well.
> But that means running that as well unless I can work out what it is doing
> on a command level. And IIUC handrake is using ffmpeg anyway under the
> hood...

I've just done something similar.

I get get_iplayer to download the files (tvbest with 50fps) and then
dump them into a folder, I then have another script that's going
through those files in that folder and converting them to x265 and
reducing the fps to 25. No issues so far, and results in a filesize of
around 350MB/per hour of TV.

It's using HandbrakeCLI (and fwiw Handbrake doesn't always use ffmpeg,
it also has other encoders built in), I'm encoding to x265 for the
smaller filesize but you can tell it to use x264 instead.

Only downside is the x265 encoding takes a while, pretty much double
the length of the video time on my system, about 1hr45 encoding for a
50-60 minute video, but hey ho, it works, and it's running in the
background via cron so it will spit out the videos when done - it's
not like I was in a hurry to watch them anyway...

For anyone interested the script is here...
https://pastebin.com/cGSxbSPK

[Ralph Corderoy]

Hi Jim,

> > You could use get_iplayer's --command option to run a command to
> > move each final file off tmpfs as the download is finished. Its
> > --output affects all the intermediate files too, AIUI.
>
> The challenge for me is to work out how to get the fetched file to go
> onto the tmpfs

Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
`--output /tmp' and you should see its intermediate files, and the final
file, only appear there. `--command' could then move that final file to
the SSD, or run a conversion command that writes to the SSD and then
removes the tmpfs input.

As for altering the ffmpeg command that get_iplayer is using, I'm not
sure that's worthwhile? It isn't doing any transcoding, just changing
the container format, or splicing in better audio, that kind of thing.
So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second
lossy one that you'd prefer to combine with the first. I could be
wrong, not knowing how to have ffmpeg do this conversion. When you find
out, let the list know. :-)

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Owen Smith]

What do you mean this isn't a lossy transcoding? How can ffmpeg go from 50fps to 25fps without losing anything? The frames are not all complete frames, software can't just throw alternate frames away. Well it could, but the only way to do that is a full H.264 decode, then discard alternate frames, then a full H.264 encode again which is going to involve loss.

Most frames are not fully present in the original stream, they are interpolated from previous and subsequent frames. You can't throw any of those away, because other frames are interpolated from them. It would need to be a very special original encode which had all even frames only interpolated from other even frames and ditto for odd frames to allow alternate frames to be discarded. And a special encode like that would bloat the file size substantially, almost doubling it I would expect.

--
Owen Smith <***@cantab.net>
Cambridge, UK

> On 2 May 2018, at 16:20, Ralph Corderoy <***@inputplus.co.uk> wrote:
>
> Hi Jim,
>
>>> You could use get_iplayer's --command option to run a command to
>>> move each final file off tmpfs as the download is finished. Its
>>> --output affects all the intermediate files too, AIUI.
>>
>> The challenge for me is to work out how to get the fetched file to go
>> onto the tmpfs
>
> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there. `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.
>
> As for altering the ffmpeg command that get_iplayer is using, I'm not
> sure that's worthwhile? It isn't doing any transcoding, just changing
> the container format, or splicing in better audio, that kind of thing.
> So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second
> lossy one that you'd prefer to combine with the first. I could be
> wrong, not knowing how to have ffmpeg do this conversion. When you find
> out, let the list know. :-)
>
> --
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
>
> _______________________________________________
> get_iplayer mailing list
> ***@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

[Peter S Kirk]

Many posts back it was mentioned they are not true 50fps, instead each
frame from a 25fps is duplicated merely to allow BBC to boast about 50fps
streaming.

On 2 May 2018 at 19:07, Owen Smith Owen Smith <***@cantab.net>
wrote:

> What do you mean this isn't a lossy transcoding? How can ffmpeg go from 50fps to 25fps without losing anything? The
> frames are not all complete frames, software can't just throw alternate frames away. Well it could, but the only way
> to do that is a full H.264 decode, then discard alternate frames, then a full H.264 encode again which is going to
> involve loss.
>
> Most frames are not fully present in the original stream, they are interpolated from previous and subsequent frames.
> You can't throw any of those away, because other frames are interpolated from them. It would need to be a very special
> original encode which had all even frames only interpolated from other even frames and ditto for odd frames to allow
> alternate frames to be discarded. And a special encode like that would bloat the file size substantially, almost
> doubling it I would expect.

[Owen Smith]

That doesn't make any difference to the H.264 encoding. At best if you get lucky every odd frame will be marked as being interpolated from the preceding even frame and then no actual changes within the frame. But all it takes is for the encoder to slip by a frame somewhere or to encode an I frame (a complete standalone frame) in the even or odd half that you are trying to discard every one of and then you are screwed.

And if the 50fps file genuinely does have every other frame saying "I'm identical to the previous frame" the stripping these won't save much on the file size. The fact that people complain the 50fps files are double the size of the same resolution 25fps files implies to me this isn't how it has been done. Even if the BBC have only created motion interpolated frames from a 25fps source, they are still encoded in the video stream and you still can't casually toss alternate frames without screwing up the I-P-B frame interpolation.

--
Owen Smith <***@cantab.net>
Cambridge, UK

> On 2 May 2018, at 19:43, Peter S Kirk <***@isauk.biz> wrote:
>
> Many posts back it was mentioned they are not true 50fps, instead each
> frame from a 25fps is duplicated merely to allow BBC to boast about 50fps
> streaming.
>
> On 2 May 2018 at 19:07, Owen Smith Owen Smith <***@cantab.net>
> wrote:
>
>> What do you mean this isn't a lossy transcoding? How can ffmpeg go from 50fps to 25fps without losing anything? The
>> frames are not all complete frames, software can't just throw alternate frames away. Well it could, but the only way
>> to do that is a full H.264 decode, then discard alternate frames, then a full H.264 encode again which is going to
>> involve loss.
>>
>> Most frames are not fully present in the original stream, they are interpolated from previous and subsequent frames.
>> You can't throw any of those away, because other frames are interpolated from them. It would need to be a very special
>> original encode which had all even frames only interpolated from other even frames and ditto for odd frames to allow
>> alternate frames to be discarded. And a special encode like that would bloat the file size substantially, almost
>> doubling it I would expect.
>
>
>
> _______________________________________________
> get_iplayer mailing list
> ***@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

[Ralph Corderoy]

Hi Owen,

> What do you mean this isn't a lossy transcoding?

Is that aimed at me?

Perhaps if you didn't top post, and instead wrote that under a quote of
mine I'd know to which bit of the two ffmpeg invocations you were
referring! :-)

> How can ffmpeg go from 50fps to 25fps without losing anything?

I don't think it can, and didn't suggest it could. It is lossy. I said
the first, default, one wasn't, and that therefore it wasn't worth
combining this extra, 50->25, one with it, as you would want to if both
were lossy.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Owen Smith]

I'm replying on an iPad, which makes large scale editing of email replies a huge amount of work. So I top post, because it takes only a tenth of the time. If Apple made it only cost me double the amount of time to reply properly I'd do it.

I've been mystified for a while why people talked about "dropping every other frame" as if it were trivial to do, and an email earlier in this chain looked like someone was trying to do that again. I was explaining why that simply is not possible in the general case.

--
Owen Smith <***@cantab.net>
Cambridge, UK

> On 2 May 2018, at 22:33, Ralph Corderoy <***@inputplus.co.uk> wrote:
>
> Hi Owen,
>
>> What do you mean this isn't a lossy transcoding?
>
> Is that aimed at me?
>
> Perhaps if you didn't top post, and instead wrote that under a quote of
> mine I'd know to which bit of the two ffmpeg invocations you were
> referring! :-)
>> How can ffmpeg go from 50fps to 25fps without losing anything?
>
> I don't think it can, and didn't suggest it could. It is lossy. I said
> the first, default, one wasn't, and that therefore it wasn't worth
> combining this extra, 50->25, one with it, as you would want to if both
> were lossy.
>
> --
> Cheers, Ralph.
> https://plus.google.com/+RalphCorderoy
>
> _______________________________________________
> get_iplayer mailing list
> ***@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/get_iplayer

[RS]

On 02/05/18 23:21, Owen Smith wrote:

>
> I've been mystified for a while why people talked about "dropping every other frame" as if it were trivial to do, and an email earlier in this chain looked like someone was trying to do that again. I was explaining why that simply is not possible in the general case.
>

If I have caused confusion by talking about dropping alternate frames I
apologise. I had come across some posts in another forum which
suggested it could be done, but I now recognise I was wrong. What makes
it worse is that I have since come across a thread in this listserver
from two years ago where I was asking exactly the same questions, and
Vangelis pointed out I was wrong and directed me to a Wikipedia article
on H.264.

It is not possible to change the frame rate using -c:v=copy in ffmpeg;
it is necessary to re-encode which is why it takes so long. Inevitably
there will be losses, added to which the codecs available to us may be
inferior to those used by the BBC.

I think it was Nick Payne who said he had experimented with re-encoding
in HEVC (H.265) and found that the file size was the same for 25fps as
it was for 50fps, which led him to conclude that frames were being
duplicated to achieve 50fps.

The broadcast signal (at least on satellite) is 1920x1080i at 25fps
usually with two audio streams, AC3 and NAR. If the broadcast signal is
recorded, the resultant file size is about 3GByte/h.

The BBC'S explanation of what it does with the broadcast signal is, "The
Elemental encoders are used to convert the 1920x1080 interlaced content
to 960x540 for progressive encoding at 50fps."

It also says, "The 50fps, 1280x720 profile, however, will be available
to those with 5Mbit/s broadband connections." but it does not explain
where it comes from or why it cannot generate a 1280x720p 25fps profile.

Best wishes
Richard

[Nick Payne]

On 3/05/2018 9:24 AM, RS wrote:
> On 02/05/18 23:21, Owen Smith wrote:
>
>>
>> I've been mystified for a while why people talked about "dropping
>> every other frame" as if it were trivial to do, and an email earlier
>> in this chain looked like someone was trying to do that again. I was
>> explaining why that simply is not possible in the general case.
>>
>
> If I have caused confusion by talking about dropping alternate frames
> I apologise.  I had come across some posts in another forum which
> suggested it could be done, but I now recognise I was wrong.  What
> makes it worse is that I have since come across a thread in this
> listserver from two years ago where I was asking exactly the same
> questions, and Vangelis pointed out I was wrong and directed me to a
> Wikipedia article on H.264.
>
> It is not possible to change the frame rate using -c:v=copy in ffmpeg;
> it is necessary to re-encode which is why it takes so long. 
> Inevitably there will be losses, added to which the codecs available
> to us may be inferior to those used by the BBC.
>
> I think it was Nick Payne who said he had experimented with
> re-encoding in HEVC (H.265) and found that the file size was the same
> for 25fps as it was for 50fps, which led him to conclude that frames
> were being duplicated to achieve 50fps.

Yes, I had some of the coverage of the UK snooker championships where
they provided HLS downloads @ 1280x720 25fps and other coverage was only
available as HVF downloads @ 1280x720 50fps. When I ran these both
through Handbrake with identical settings to convert to HEVC, retaining
the frame rate of the downloaded files, the size reduction for the 50fps
downloads was about twice that for the 25fps downloads, and the size of
the files output by Handbrake was proportional to the length of the
program and not the frame rate. e.g.

4h58m match: D/L size 5.03Gb @ 25fps, output from Handbrake was 1.55Gb
4h44m match: D/L size 10.2Gb @ 50fps, output from Handbrake was 1.48Gb

Nick

[Ralph Corderoy]

Hi Nick,

> I had some of the coverage of the UK snooker championships
...
> 4h58m match: D/L size 5.03Gb @ 25fps, output from Handbrake was 1.55Gb
> 4h44m match: D/L size 10.2Gb @ 50fps, output from Handbrake was 1.48Gb

Perhaps many frames(!) of snooker coverage isn't ideal for this
comparison as it often is a static picture of the table. :-)

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Ralph Corderoy]

Hi Richard,

> I think it was Nick Payne who said he had experimented with
> re-encoding in HEVC (H.265) and found that the file size was the same
> for 25fps as it was for 50fps, which led him to conclude that frames
> were being duplicated to achieve 50fps.

ffmpeg can produce PNGs, one per frame, and convert only a few specific
seconds to avoid tens of thousands of them, and then adjacent frames
could be compared in pairs to see if they are indeed identical; compare
`ab', `bc', `cd'...

There's a couple of variants for comparing images.

compare foo.png bar.png x:
gm compare -highlight-style assign foo.png bar.png -file x:

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[RS]

On 03/05/18 11:41, Ralph Corderoy wrote:
>
> ffmpeg can produce PNGs, one per frame, and convert only a few specific
> seconds to avoid tens of thousands of them, and then adjacent frames
> could be compared in pairs to see if they are indeed identical; compare
> `ab', `bc', `cd'...
>
> There's a couple of variants for comparing images.
>
> compare foo.png bar.png x:
> gm compare -highlight-style assign foo.png bar.png -file x:
>
Hi Ralph

Would you first need to convert the H.264 or H.265 to raw video? If one
PNG is of an I-frame and the next is a P-frame or B-frame they are bound
to be different.

Best wishes
Richard

[Ralph Corderoy]

Hi Richard,

> > ffmpeg can produce PNGs, one per frame, and convert only a few
> > specific seconds to avoid tens of thousands of them
>
> Would you first need to convert the H.264 or H.265 to raw video? If
> one PNG is of an I-frame and the next is a P-frame or B-frame they are
> bound to be different.

No, AIUI ffmpeg(1) produces a PNG that's the whole assembled picture,
just as if one hit `Pause' when watching TV. Any indication of how that
was built up from the input video encoding is lost.

ffmpeg -i foo.mp4 -t 1 foo-%04d.png

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Steve Dodd]

On Thu, May 3, 2018 at 12:24 AM, RS <***@zoho.com> wrote:

> I think it was Nick Payne who said he had experimented with re-encoding in HEVC (H.265) and found that the file size was the same for 25fps as it was for 50fps, which led him to conclude that frames were being duplicated to achieve 50fps.

Is it possible it depends on the source material? From the BBC quote
earlier it sounds like their "source" material is still mostly HD
interlaced, but perhaps some of their sources are all also
progressive, and those ones get doubled identical frames (which might
be a logical result of feeding 25p material into something that's
designed to interpolate interlaced fields into frames)?

S.

[RS]

On 03/05/18 12:18, Steve Dodd wrote:

> Is it possible it depends on the source material? From the BBC quote
> earlier it sounds like their "source" material is still mostly HD
> interlaced, but perhaps some of their sources are all also
> progressive, and those ones get doubled identical frames (which might
> be a logical result of feeding 25p material into something that's
> designed to interpolate interlaced fields into frames)?
>
The satellite broadcasts I have seen from the BBC and other broadcasters
are 1920x1080i25 for HD and 720x576i25 or 704x576i25 for SD. Everything
that is broadcast is in an interlaced format at some stage, however it
is generated. Some programmes are only produced for the iPlayer, so
they may be generated in a different way.

Best wishes
Richard

[Christopher Woods]

On 3 May 2018 21:38:23 RS <***@zoho.com> wrote:

> On 03/05/18 12:18, Steve Dodd wrote:
>
>> Is it possible it depends on the source material? From the BBC quote
>> earlier it sounds like their "source" material is still mostly HD
>> interlaced, but perhaps some of their sources are all also
>> progressive, and those ones get doubled identical frames (which might
>> be a logical result of feeding 25p material into something that's
>> designed to interpolate interlaced fields into frames)?
> The satellite broadcasts I have seen from the BBC and other broadcasters
> are 1920x1080i25 for HD and 720x576i25 or 704x576i25 for SD. Everything
> that is broadcast is in an interlaced format at some stage, however it
> is generated. Some programmes are only produced for the iPlayer, so
> they may be generated in a different way.


A handful of HD is still 1440x1080i25 :( the non-square pixel format is a
great cheat for bandwidth saving. The chroma compression on current
generation terrestrial broadcasting also makes everything look bad. Once
you watch uncompressed HD-SDI you never want anything else :) Some SD is
way below what we would accept as SD resolution as well, all in the name of
cost saving.


Steve is right on the doubled technique, this is the filmic look you see on
documentaries and dramas. That source material will be likely be captured
as progressive frames, then upconverted to 50 fields per second,
'progressive segmented frame' format (PsF) in the edit.

The same image is stored across every two fields that constitute one frame,
meaning you're being shown one effective whole full resolution image, 25
times a second.

With standard interlaced footage, you're being shown overlapping effective
'images' at half-resolution, 50 times a second. Persistence of vision
(well, nowadays, flat panel processing circuitry) bob deinterlaces the
fields to produce 50 images per second.


However, TVs sometimes do this artificially by taking true progressive
sources and interpolating the footage to a higher refresh rate (100/200
Hz). This inherently looks dire and should be disabled.

On some TVs with poorly written image DSP (or badly chosen settings) you'll
see the picture suddenly go 'filmic' as the deinterlace method changes to
blend, which literally blends together pairs of fields to produce 25 frames.

Some explainers of fields, frames and progressive segmented format interlaced:

https://en.m.wikipedia.org/wiki/Progressive_segmented_frame

https://wolfcrow.com/blog/understanding-terminology-progressive-segmented-frames-psf/
https://wolfcrow.com/blog/understanding-terminology-progressive-frames-interlaced-frames-and-the-field-rate/

[Lucy Walker]

On 07/05/2018 17:09, Christopher Woods wrote:
>
> Steve is right on the doubled technique, this is the filmic look you
> see on documentaries and dramas. That source material will be likely
> be captured as progressive frames, then upconverted to 50 fields per
> second, 'progressive segmented frame' format (PsF) in the edit.
>

At least one "yoof soap" shoots 1080p25 shuttered to 1/50th - looks like
shit, but the head of cameras is a fucking moron. No more processing
than that

[Dave Lambley]

On 3 May 2018 at 00:24, RS <***@zoho.com> wrote:
> On 02/05/18 23:21, Owen Smith wrote:
>
>>
>> I've been mystified for a while why people talked about "dropping every
>> other frame" as if it were trivial to do, and an email earlier in this chain
>> looked like someone was trying to do that again. I was explaining why that
>> simply is not possible in the general case.
>>
>
> If I have caused confusion by talking about dropping alternate frames I
> apologise. I had come across some posts in another forum which suggested it
> could be done, but I now recognise I was wrong. What makes it worse is that
> I have since come across a thread in this listserver from two years ago
> where I was asking exactly the same questions, and Vangelis pointed out I
> was wrong and directed me to a Wikipedia article on H.264.
>
> It is not possible to change the frame rate using -c:v=copy in ffmpeg; it is
> necessary to re-encode which is why it takes so long. Inevitably there will
> be losses, added to which the codecs available to us may be inferior to
> those used by the BBC.
>
> I think it was Nick Payne who said he had experimented with re-encoding in
> HEVC (H.265) and found that the file size was the same for 25fps as it was
> for 50fps, which led him to conclude that frames were being duplicated to
> achieve 50fps.
>
> The broadcast signal (at least on satellite) is 1920x1080i at 25fps usually
> with two audio streams, AC3 and NAR. If the broadcast signal is recorded,
> the resultant file size is about 3GByte/h.
>
> The BBC'S explanation of what it does with the broadcast signal is, "The
> Elemental encoders are used to convert the 1920x1080 interlaced content to
> 960x540 for progressive encoding at 50fps."

Searching turns up this,

http://dpp-assets.s3.amazonaws.com/wp-content/uploads/specs/bbc/TechnicalDeliveryStandardsBBCFile.pdf

... which tells you that 25fps interlaced is what programme makers
will be supplying for HD programmes.

> It also says, "The 50fps, 1280x720 profile, however, will be available to
> those with 5Mbit/s broadband connections." but it does not explain where it
> comes from or why it cannot generate a 1280x720p 25fps profile.

They would remove the interlace to bring the framerate to 50fps before
encoding. There are many methods of doing so, ffmpeg can do this for
example.

Dave

[RS]

On 04/05/18 23:50, Dave Lambley wrote:

>
> They would remove the interlace to bring the framerate to 50fps before
> encoding. There are many methods of doing so, ffmpeg can do this for
> example.
>
This article

https://superuser.com/questions/253691/how-to-convert-108050i-72050p-using-ffmpeg

says, "Just like interlacing progressive content divides each frame into
2 fields, thus doubling the frame rate, the common method of
deinterlacing is to combine each 2 fields into 1 frame, which reduces
the rate by 2, thus taking 50 into 25fps. You can of course double each
final frame, but that does not provide any benefit."

Do you disagree?

Best wishes
Richard

[Dave Lambley]

On 5 May 2018 at 00:29, RS <***@zoho.com> wrote:
> On 04/05/18 23:50, Dave Lambley wrote:
>
>>
>> They would remove the interlace to bring the framerate to 50fps before
>> encoding. There are many methods of doing so, ffmpeg can do this for
>> example.
>>
> This article
>
> https://superuser.com/questions/253691/how-to-convert-108050i-72050p-using-ffmpeg
>
> says, "Just like interlacing progressive content divides each frame into 2
> fields, thus doubling the frame rate, the common method of deinterlacing is
> to combine each 2 fields into 1 frame, which reduces the rate by 2, thus
> taking 50 into 25fps. You can of course double each final frame, but that
> does not provide any benefit."
>
> Do you disagree?

eg., https://ffmpeg.org/ffmpeg-filters.html#bwdif (and other) filters
can give you good quality 50fps frames, without duplication, from
interlaced 25fps video. I have no idea what the BBC use, but they have
plenty of choice.

Dave

[Jim web]

In article
<CAPCK=o8mEqu3oE1KK5O3YOpsD=czK52k4-J-N6=og-SzM=***@mail.gmail.com>, Dave
Lambley <***@lambley.me.uk> wrote:

> Searching turns up this,

> http://dpp-assets.s3.amazonaws.com/wp-content/uploads/specs/bbc/TechnicalDeliveryStandardsBBCFile.pdf

> ... which tells you that 25fps interlaced is what programme makers will
> be supplying for HD programmes.

Thats' quite interesting. It seems to confirm a feeling I had that the
input might often be 25fps. Is there a conflation or mix up between frames
and fields somewhere?

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[Jim web]

In article <***@orac.inputplus.co.uk>, Ralph
Corderoy <***@inputplus.co.uk> wrote:
> >
> > The challenge for me is to work out how to get the fetched file to go
> > onto the tmpfs

> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there. `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.

Thanks. :-)

> As for altering the ffmpeg command that get_iplayer is using, I'm not
> sure that's worthwhile? It isn't doing any transcoding, just changing
> the container format, or splicing in better audio, that kind of thing.
> So your `lossy' slow-down ffmpeg from 50 fps to 25 fps won't be a second
> lossy one that you'd prefer to combine with the first. I could be
> wrong, not knowing how to have ffmpeg do this conversion. When you find
> out, let the list know. :-)

My concern is that I only have a total of 8GB of ram at present on the
machine, and the 1280x720 50fps files tend to come in at 2GB or more per
hour.

To save time and avoid running past 9am (and entering metered time) it
would be quickest to do all the fetches first. But this will mean more
files that I can store on ram on some days. So the wish would be to then
shift or process material and put it on hd.

As things are, I could then have gip+ffmpeg generate the 50 fps mp4 files
(i.e. in the current form) onto hd. This saves some hd wear but means I now
have those (big) files on hd before I then run a 50 -> 25 fps process to
create smaller ones (which will be akin in size to what I got from the
25fps fetching). The result using this approach means I've still done the
fetching as quickly as possible, but have about three times as much data
written to hd at some point.

It would be nice to do the process file by file and in each case only the
'final' 25 fps version gets written to hd. This means the same hd use as
previously. But that either means I need enough ram to hold all the 'temp'
files (masses of ram needed) or doing the process item by item. (slow)

I could use spinning rust I guess. but that seems messy. Otherwise it seems
to want me to buy and fit a lot of ram. :-)

So at present I'm wondering and experimenting. 8-]

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[iz]

On 2 May 2018 at 16:49, Jim web <***@audiomisc.co.uk> wrote:
>
> As things are, I could then have gip+ffmpeg generate the 50 fps mp4 files
> (i.e. in the current form) onto hd. This saves some hd wear but means I now
> have those (big) files on hd before I then run a 50 -> 25 fps process to
> create smaller ones (which will be akin in size to what I got from the
> 25fps fetching). The result using this approach means I've still done the
> fetching as quickly as possible, but have about three times as much data
> written to hd at some point.

You may be able to limit the whole process to only two file writes
with a little scripting and XML parsing. Use --raw with GiP to save
the .ts file directly to drive or to drive via ram disk. The ram disk
probably wouldn't be of much use since I doubt your downloads are
limited by disk write speed. Also use --metadata to save programme
metadata in an XML file and --thumbnail to save the cover art in a JPG
file. Write a script to re-encode the file to 25fps MP4 and at the
same time add metadata tags with ffmpeg, using the XML file and JPG
file as input to construction of the ffmpeg command string. You'll
need ffmpeg 4.0 or higher to add cover art to MP4.

[Jim web]

In article
<CAF_ssT_0x-***@mail.gmail.com>, iz
<***@gmail.com> wrote:
> On 2 May 2018 at 16:49, Jim web <***@audiomisc.co.uk> wrote:
> >
> > As things are, I could then have gip+ffmpeg generate the 50 fps mp4
> > files (i.e. in the current form) onto hd. This saves some hd wear but
> > means I now have those (big) files on hd before I then run a 50 -> 25
> > fps process to create smaller ones (which will be akin in size to what
> > I got from the 25fps fetching). The result using this approach means
> > I've still done the fetching as quickly as possible, but have about
> > three times as much data written to hd at some point.

> You may be able to limit the whole process to only two file writes with
> a little scripting and XML parsing.

FWIW as things stand I run the processes via a simple proglet I wrote in
'C'. (I use it as a ROX-Filer app, but it can also work via CLI.) I can
drag-and-drop a text file onto the program's icon. The file specifies a
'mode' and a pid per line, and the program then proceeds to chug though
them as I make breakfast. If needed, the file I give it could also say if
any particular reprocessing need be done 'now' or left to later, etc.

So I'd almost certainly do this via a 'C' approach because I've become used
to 'C' over the years. The trick is knowing what commands to launch, etc.

Modifying where output goes, making the output -raw, etc, is OK as I know
how to do that - once I've sorted out unknowns like if the -raw is going to
ramdisc or somewhere else, when reprocessing is done, etc.

Thus, yes, I am thinking it should be possible to do this via running 'one
program'. But...

Truth is, having finally allowed people to prise FORTRAN out of my cold
dead hands a decade or more ago, I'm now settled on 'C'. :-)

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[Budge]

On 20/05/18 23:32, SquarePenguin wrote:
> On 20/05/2018 23:23, Budge wrote:
>> Either ffmpeg or Handbrake will do, whichever works!
>
> There is a post on the forum where someone explains how they use
> Handbrake to do it:
>
> https://forums.squarepenguin.co.uk/thread-1786-post-7781.html#pid7781
>
> There is also this, using ffmpeg, (but it's a little simplistic):
>
> https://forums.squarepenguin.co.uk/thread-1765.html?highlight=50fps
>
> Which leads here (which shows more options):
>
> https://trac.ffmpeg.org/wiki/Encode/H.264

Hi and many thanks for the links.
I had seen and read all but the first before posting but not fully
understood what I should be doing. I am not lazy, just too busy to
concentrate on this at present and was hoping to be able to copy a
suitable command line example. Will return to topic and post my results
as soon as I can but if others are ahead of me please do share.
Thanks again,
budge

[Ralph Corderoy]

Hi Jim,

> My concern is that I only have a total of 8GB of ram at present on the
> machine, and the 1280x720 50fps files tend to come in at 2GB or more
> per hour.

Yes, you'd need to finish one download off, clearing RAM disk, before
starting the next.

> To save time and avoid running past 9am (and entering metered time) it
> would be quickest to do all the fetches first.

I too have metered broadband, but have automated use of it begin at
00:03 using timeout(1) to ensure it is killed by my 08:00 cut-off time
if it didn't complete for some reason. Have you considered something
similar, perhaps using get_iplayer's `--pvr-queue', etc., to add
programmes of interest, and then telling it to `run' some or all of the
queue items?

> but have about three times as much data written to hd at some point.

Yes, with my preferred choices, get_iplayer's writes to disk would be

downloaded video
downloaded audio
video and audio combined into MP4
tagged MP4

so 3(v+a), and I'm not converting to 25 fps.

> I could use spinning rust I guess. but that seems messy.

Do you have it already available in the machine to use? That seems an
easy compromise.

You could also look at your SSD's specifications and whether you're
striving too hard to avoid what might not be a problem with modern ones
and their built-in wear levelling.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[Jim web]

In article <***@orac.inputplus.co.uk>, Ralph
Corderoy <***@inputplus.co.uk> wrote:
> > The challenge for me is to work out how to get the fetched file to go
> > onto the tmpfs

> Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> `--output /tmp' and you should see its intermediate files, and the final
> file, only appear there. `--command' could then move that final file to
> the SSD, or run a conversion command that writes to the SSD and then
> removes the tmpfs input.

the df command listed various 'locations'. These included

tmpfs 815696 1352 814334 /run

none 4078460 0 4078460 /run/shm

none 102400 8 102392 /run/usr

Inside /run/usr I did find a directory for my user id and could use that as
the user. However I don't know what the /run/shm is for, so need to do some
finding out...

I did set up an explict ramdisc on another machine ages ago by adding a
line to the /etc/fstab file. But that fixed the size to 256 MB, which in
this context is tiny. System monitor says I have 8GB of ram.

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html

[Ralph Corderoy]

Hi Jim,

> > Well, `df -t tmpfs' will probably show /tmp is a tmpfs so you could
> > `--output /tmp' and you should see its intermediate files, and the
> > final file, only appear there. `--command' could then move that
> > final file to the SSD, or run a conversion command that writes to
> > the SSD and then removes the tmpfs input.
>
> the df command listed various 'locations'. These included
>
> tmpfs 815696 1352 814334 /run
> none 4078460 0 4078460 /run/shm
> none 102400 8 102392 /run/usr

/run/shm is half your RAM, as I'd expect. I'd also expect /tmp to be
there. What does `df -T /tmp' show? Perhaps that's using SSD for all
temporary files and you might not want that given your concerns.

> Inside /run/usr I did find a directory for my user id and could use
> that as the user.

But it's deliberately constrained to be small.

> However I don't know what the /run/shm is for, so need to do some
> finding out...

POSIX's communication methods of shared memory and semaphores;
shm_overview(7), and sem_overview(7). Again, not what you want.

> I did set up an explict ramdisc on another machine ages ago by adding
> a line to the /etc/fstab file. But that fixed the size to 256 MB,
> which in this context is tiny. System monitor says I have 8GB of ram.

That's the right idea. Here, the system has /tmp be a tmpfs that is
half the RAM, like the `shm' one above. They share one single half.
`systemctl status tmp.mount' is what creates it on some systems these
days, not an /etc/fstab entry.

--
Cheers, Ralph.
https://plus.google.com/+RalphCorderoy

[RS]

On 30/04/18 13:56, Jim web wrote:
> I've been discussing the 'loss' of the 1280x720 25fps version with someone
> at the BBC.
>
> IIUC this stemmed from 'Red Bee' days of yore, and until recently people at
> the BBC had thought they had stopped it long ago. Someone apparently
> noticed recently that it was still available. And then actually disabled
> it. Hence the mysterious recent ending of its availablity.
>
> I have made the point that the 1280x720 25fps version is useful for people
> with a content 'cap' problem, etc, so will be missed because the 50fps
> version means somewhat bigger files and/or stream rates. But I doubt this
> will cause a rethink.
>
We have known for some time that the BBC was going to stop using Flash
at some stage. Dinky anticipated that change and get_iplayer no longer
supports Flash. The last version with Flash capability was v2.99.

HLS is a much more recent innovation than Flash. get_iplayer now has
its own built-in downloader which downloads HLS 2 or 3 times as fast as
Flash. HLS has been referred to as a legacy mode so it was always
likely that it would be removed eventually. The surprise was that HLS
was removed at the same time as Flash.

It is unrealistic to expect the BBC to restore HLS or Flash.

I have not seen any comments here which suggest that anyone is unhappy
with HVF. get_iplayer uses the same built-in downloader as for HLS.
The BBC's full list of HVF modes is set out in a table in this document.

https://www.bbc.co.uk/rd/blog/2015-07-the-development-of-new-video-factory-profiles-for-bbc-iplayer

What I think a number of people here would like to see is the addition
of one more High H.264 profile mode to that list, namely 1280x720p 25fps.

The BBC refers to viewing tests it has carried out, "using a range of
content and included clips, from popular shows such as Strictly Come
Dancing, East Enders and Top Gear."

It concludes, "At around 3 Mbit/s a 960x540 profile at 50fps will be
made available to Connected TVs and set top boxes. The Elemental
encoders are used to convert the 1920x1080 interlaced content to 960x540
for progressive encoding at 50fps. Although the 960x540 profile has a
reduced spatial resolution compared to the outgoing 1280x720 at 25fps,
subjective assessments shows it delivers significantly better pictures
on TV screens across a wide range of popular content (such as EastEnders
and Top Gear) due to its higher frame rate. The 50fps, 1280x720 profile,
however, will be available to those with 5Mbit/s broadband connections.

"Additional lower bit-rate profiles will be made available to computers
on wifi. This enables video playback to continue, even when an
individual's available bit-rate is reduced by users sharing a
connection. These profiles will also cater for similar bit-rate
restrictions on public wifi connections."

I accept that there is a trade off between resolution and frame rate.
With differential encoding schemes like H.264, at a higher frame rate
the changes between frames are smaller, so there is less to encode.
Even so, for the uncompressed video the resolution at 50fps would need
to be reduced to 905x510p to give the same bit rate as 1280x720p at 25fps.

For that reason I find it surprising that 960x540p at 50fps "delivers
significantly better pictures on TV screens across a wide range of
popular content (such as EastEnders and Top Gear) due to its higher
frame rate" than 1280x720p at 25fps but the BBC has done the viewing
tests and I have not.

More importantly BBC Four and to a lesser extent BBC 2 have a lot of
programmes about paintings, sculpture, architecture and nature where
there is a lot of fine detail and little motion. Intuitively such
programmes would not benefit from the higher frame rate but would
benefit from the higher resolution. There is no mention in the blog
that such programmes were included in the viewing tests, and they ought
to have been.

Best wishes
Richard

[MacFH - C E Macfarlane]

Please see below ...

On 02/05/2018 11:50, RS wrote:
>
> On 30/04/18 13:56, Jim web wrote:
>> I've been discussing the 'loss' of the 1280x720 25fps version with
>> someone
>> at the BBC.
>>
>> IIUC this stemmed from 'Red Bee' days of yore, and until recently
>> people at
>> the BBC had thought they had stopped it long ago. Someone apparently
>> noticed recently that it was still available. And then actually disabled
>> it. Hence the mysterious recent ending of its availablity.
>>
>> I have made the point that the 1280x720 25fps version is useful for
>> people
>> with a content 'cap' problem, etc, so will be missed because the 50fps
>> version means somewhat bigger files and/or stream rates. But I doubt
>> this
>> will cause a rethink.
>>
> We have known for some time that the BBC was going to stop using Flash
> at some stage.  Dinky anticipated that change and get_iplayer no
> longer supports Flash.  The last version with Flash capability was v2.99.
>
> HLS is a much more recent innovation than Flash.  get_iplayer now has
> its own built-in downloader which downloads HLS 2 or 3 times as fast
> as Flash.  HLS has been referred to as a legacy mode so it was always
> likely that it would be removed eventually.  The surprise was that HLS
> was removed at the same time as Flash.
>
> It is unrealistic to expect the BBC to restore HLS or Flash.

Yes, it' s unrealistic to expect the BBC to respond to any criticism
whatsoever, because they never have in the past whenever changes they've
introduced have broken other people's kit such as Network Media Players
and Smart TVs.

> I have not seen any comments here which suggest that anyone is unhappy
> with HVF.  get_iplayer uses the same built-in downloader as for HLS.
> The BBC's full list of HVF modes is set out in a table in this document.
>
> https://www.bbc.co.uk/rd/blog/2015-07-the-development-of-new-video-factory-profiles-for-bbc-iplayer
>
>
> What I think a number of people here would like to see is the addition
> of one more High H.264 profile mode to that list, namely 1280x720p 25fps.

Not *necessarily*  -  I think the problem is more subtle than that. See
below ...

> The BBC refers to viewing tests it has carried out, "using a range of
> content and included clips, from popular shows such as Strictly Come
> Dancing, East Enders and Top Gear."
>
> It concludes, "At around 3 Mbit/s a 960x540 profile at 50fps will be
> made available to Connected TVs and set top boxes. The Elemental
> encoders are used to convert the 1920x1080 interlaced content to
> 960x540 for progressive encoding at 50fps. Although the 960x540
> profile has a reduced spatial resolution compared to the outgoing
> 1280x720 at 25fps, subjective assessments shows it delivers
> significantly better pictures on TV screens across a wide range of
> popular content (such as EastEnders and Top Gear) due to its higher
> frame rate. The 50fps, 1280x720 profile, however, will be available to
> those with 5Mbit/s broadband connections.

That's what I see as the problem, not a single natural history or art or
historical documentary mentioned in the testing mix.  For such
programmes, I would suggest that often an increase in spatial resolution
would be preferable to an increase in temporal resolution.  I don't care
how popular the above programmes are, I've not watched one of them in
more than 20 years, and the other two ever.  The BBC is a public service
broadcaster, and shouldn't only be thinking about what is best for
populist programming.  They should be doing what is best for the content
of a particular programme, or if that is too complicated and difficult,
then, yes, let us have the choice of a 1280x720x25 profile, and let us
choose which to download.

> I accept that there is a trade off between resolution and frame rate.
> With differential encoding schemes like H.264, at a higher frame rate
> the changes between frames are smaller, so there is less to encode.
> Even so, for the uncompressed video the resolution at 50fps would need
> to be reduced to 905x510p to give the same bit rate as 1280x720p at
> 25fps.
>
> For that reason I find it surprising that 960x540p at 50fps "delivers
> significantly better pictures on TV screens across a wide range of
> popular content (such as EastEnders and Top Gear) due to its higher
> frame rate" than 1280x720p at 25fps but the BBC has done the viewing
> tests and I have not.
>
> More importantly BBC Four and to a lesser extent BBC 2 have a lot of
> programmes about paintings, sculpture, architecture and nature where
> there is a lot of fine detail and little motion. Intuitively such
> programmes would not benefit from the higher frame rate but would
> benefit from the higher resolution.  There is no mention in the blog
> that such programmes were included in the viewing tests, and they
> ought to have been.

Yes, agreed (I wrote the above before I'd read your post entirely, which
was probably not best practice, for which apologies, but I'm in
something of a hurry).

But there is also another elephant in the room, which is the capacity of
their server system to deliver whatever they choose to do.  There's
little point in increasing either spatial or temporal resolution if the
resulting download will be compressed to hell on the way out of the
servers.  I've seen even HD programmes break up into squares, possibly
because the system can't cope with the throughput at times of peak
demand, though I ought to point out in fairness that on at least one
occasion the squares seem to be in the source programme, because
downloading it a second time gave an identical result!

Regards.

[Jim web]

In article <60733938-d7ea-4db5-bd1d-***@zoho.com>, RS
<***@zoho.com> wrote:


> On 30/04/18 13:56, Jim web wrote:
> > I've been discussing the 'loss' of the 1280x720 25fps version with
> > someone at the BBC.
> >
> > IIUC this stemmed from 'Red Bee' days of yore, and until recently
> > people at the BBC had thought they had stopped it long ago. Someone
> > apparently noticed recently that it was still available. And then
> > actually disabled it. Hence the mysterious recent ending of its
> > availablity.
> >
> > I have made the point that the 1280x720 25fps version is useful for
> > people with a content 'cap' problem, etc, so will be missed because
> > the 50fps version means somewhat bigger files and/or stream rates. But
> > I doubt this will cause a rethink.
> >


> HLS is a much more recent innovation than Flash. get_iplayer now has
> its own built-in downloader which downloads HLS 2 or 3 times as fast as
> Flash. HLS has been referred to as a legacy mode so it was always
> likely that it would be removed eventually. The surprise was that HLS
> was removed at the same time as Flash.

> It is unrealistic to expect the BBC to restore HLS or Flash.

Understood. I've probably been conflating different issues

[snip BBC's comments]


> For that reason I find it surprising that 960x540p at 50fps "delivers
> significantly better pictures on TV screens across a wide range of
> popular content (such as EastEnders and Top Gear) due to its higher
> frame rate" than 1280x720p at 25fps but the BBC has done the viewing
> tests and I have not.

I'm also puzzled by this conclusion by the BBC. But I haven't viewed more
than a few of the 960x540 examples, so my findings may not be typical.

> More importantly BBC Four and to a lesser extent BBC 2 have a lot of
> programmes about paintings, sculpture, architecture and nature where
> there is a lot of fine detail and little motion. Intuitively such
> programmes would not benefit from the higher frame rate but would
> benefit from the higher resolution. There is no mention in the blog
> that such programmes were included in the viewing tests, and they ought
> to have been.

I'd agree. I'll see if I can pass forward some of these points. I need
to be able to contact someone further 'up the stack' to ask more
'policy' questions I guess. I know more about the radio side, so
I don't know how I'll get on with the tv side.

[edit] Just sent an email making some of the points. I also added
a comment of my own that I'd *much* prefer "Sky at Night" as the
1280x720 25fps than a lower resolution. Time will tell if I manage
to get someone 'higher up' to be willing to discuss this with me.

Jim

--
Electronics https://www.st-andrews.ac.uk/~www_pa/Scots_Guide/intro/electron.htm
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html
biog http://jcgl.orpheusweb.co.uk/history/ups_and_downs.html
Audio Misc http://www.audiomisc.co.uk/index.html