Author Topic: Merging weird behavior  (Read 4755 times)

Offline rublev1360

  • Newbie
  • *
  • Posts: 31
  • Karma: +2/-0
    • andreabianco.eu
Merging weird behavior
« on: January 15, 2012, 11:35:23 pm »
Hi there, I thought of posting this on the IRC channel but since I'm not getting feedback there I try here.

This is the situation:

- the image has more than 2 layers
- one layer has a blending mode applied that is different from normal (dodge in my case)
- the layer with the blending mode is on top of at least 1 more layer before the last one

What happens:

- on merging down the layer with the blending mode not normal I lose its blending mode and so it behaves like it is normal

To have the merge keeping the special blending mode I need to have only 1 layer below the one with the (let's say) dodge blending mode.

I hope I made myself clear. This is really bothering since it's common to have layers with dodge or burn on top of base layers with the general values, and to get the blending modes being applied correctly it's needed to first lose all the layer structure.

I'm running MyPaint 64 bit on Windows 7. Is someone else experiencing this behavior?

Offline savageorange

  • Full Member
  • ***
  • Posts: 158
  • Karma: +9/-0
Re: Merging weird behavior
« Reply #1 on: January 16, 2012, 02:22:14 am »
yes of course. Happens literally all the time. What else would you expect to happen ? (think about the mathematics involved. Is it really possible to preserve the blending mode and have the result make sense?)

Offline rublev1360

  • Newbie
  • *
  • Posts: 31
  • Karma: +2/-0
    • andreabianco.eu
Re: Merging weird behavior
« Reply #2 on: January 16, 2012, 11:43:07 am »
I painted two images to explain better the concept. The first one is made with Gimp, the second one with MyPaint. For all of the images the following is valid:

L0 is the lower layer, L1 the middle one, L2 the higher one

A = gray painting on layer L0 (set to normal)
B = gray painting on layer L1 (set to normal)
C = red painting on layer L2 (set to dodge)

I = separate layers, L2 is not present
II = separate layers, L2 is present
III = L2 merged down on L1, L0 is still separate

Gimp:




MyPaint:



As you can see Gimp when merging down is correctly applying the dodge layer blending and it is only affecting the layer below, as the "merge down" function would make you expect. MyPaint on the other hand is converting the dodge layer to a normal one before the merging, and the result is wrong and makes the layer blending modes almost useless. I hope I made myself clear :)

Offline David Revoy

  • Mypaint Dev
  • Full Member
  • *****
  • Posts: 207
  • Karma: +13/-0
    • portfolio
Re: Merging weird behavior
« Reply #3 on: January 16, 2012, 01:54:43 pm »
Your explanation about are clear. But I disagree about the Gimp 'correct behavior' in this case.
Both Mypaint and Gimp ( and all 2D softwares I used the last 10 years ) are wrong about.

Let's look at a mini concrete exemple :


1. User did a scene with a character ( a rabbit ) , and a background landscape. Both "normal blending mode" layer.
The user also added on the top a dodge/screen or whatever "layer blending mode" on the top.


2. His "layer blending mode" on top affect whole of the picture. And User is happy with it.


3. [Gimp/ behavior] Now user call a "merge down" to his "layer blending mode".
the effect will only affect the available pixel under ( the head of bunny and the crown ).
It will lead to a data loss , because the effect User liked on the grass / sky disappeared.
User is not happy.


4. [Mypaint / Krita behavior] Now user call a "merge down" to his "layer blending mode".
To avoid a data loss, the effect will only affect the available pixel under ( the head of bunny and the crown )
but also the pixel of the "layer blending mode" meeting a pure alpha transparent place are simply copied to the normal layer.
Of course, in this image case it can be not good as the sky and the grass are now totally covered.
User is not happy.


5. A proposition ? sure...  When performing a merge down layer, if the layer has a blending mode,
and so create an ambiguous situation , the merging should propose ( a modal dialog ) to :
- Affect only layer under (3, as the way of Gimp)
- Affect all layers ; a new choice described on picture (5). The user can keep the bunny and the background this way with the "layer blending mode" properly merged.

A new question happen whith the proposed system :
What happen if a "layer blending mode" as multiply , filled in red is here to teint the sky between the background and the bunny while a merge of blending mode happen on the top ? In my opinion, the merge modal dialog should also warn it will automerge all the other layer blending mode lower in the layer stack.
With the exemple it will make the red blending mode for the sky blend with the sky first, then the effect on the top also blended on multilayer.





 

Offline rublev1360

  • Newbie
  • *
  • Posts: 31
  • Karma: +2/-0
    • andreabianco.eu
Re: Merging weird behavior
« Reply #4 on: January 16, 2012, 02:54:14 pm »
Thanks for your cute & clear graphic explanation and the time you did put into it, David :)

When working with Gimp I only use layers with blending modes different from normal for specifically adding to the layer below them, since I'm aware of the fact that the merge down will only affect that layer, so I'm never in an "unhappy user" situation luckily; in MyPaint such a way of working is not possible and so I think that Gimp's way, though not perfect, is currently more versatile and better than MyPaint's one. Knowing the way the merge down works in Gimp, the user of your example could simple paint the stuff for the dodge/burn/etc on different layers and put each one on top of the one he/she wanna affect, merging them separately depending on the needings and not loosing all of the layer structure only cos he/she needs to merge down a layer!

I discovered this issue in MyPaint while working on a new concept/test, since I had a layer with blocking, one with details and one with some color variations, and I only wanted to merge the color variations on the detail layer.. which is currently not possible. So I had to merge all the image down (I want to do this project all in Mypaint) and while this is fine for a personal project it's a no-go for a project where you are requested to keep a degree of layer separation.

Your proposal is also very interesting and would deserve attention; and in meantime it would be very useful to at least implement Gimp's way, together with two or three more blending modes ;)

Offline David Revoy

  • Mypaint Dev
  • Full Member
  • *****
  • Posts: 207
  • Karma: +13/-0
    • portfolio
Re: Merging weird behavior
« Reply #5 on: January 16, 2012, 04:26:39 pm »
Thanks for your cute & clear graphic explanation and the time you did put into it, David :)

When working with Gimp I only use layers with blending modes different from normal for specifically adding to the layer below them, since I'm aware of the fact that the merge down will only affect that layer, so I'm never in an "unhappy user" situation luckily; in MyPaint such a way of working is not possible and so I think that Gimp's way, though not perfect, is currently more versatile and better than MyPaint's one. Knowing the way the merge down works in Gimp, the user of your example could simple paint the stuff for the dodge/burn/etc on different layers and put each one on top of the one he/she wanna affect, merging them separately depending on the needings and not loosing all of the layer structure only cos he/she needs to merge down a layer!

I discovered this issue in MyPaint while working on a new concept/test, since I had a layer with blocking, one with details and one with some color variations, and I only wanted to merge the color variations on the detail layer.. which is currently not possible. So I had to merge all the image down (I want to do this project all in Mypaint) and while this is fine for a personal project it's a no-go for a project where you are requested to keep a degree of layer separation.

Your proposal is also very interesting and would deserve attention; and in meantime it would be very useful to at least implement Gimp's way, together with two or three more blending modes ;)

I definitely think we are on the same page. the Gimp way to merge is indeed a better behavior than actual, and the first priority as a workaround fix would be to switch to this method for merging. And very true about this ; with this Gimp method, user can still duplicate the layer blending mode , and merge them over layer separately.
I let you fill a 'bug' on the Mypaint bugtracker to report it, and link to this discussion.

My proposition (5) would require certainly a lot more coding hours and effort... But in a way not sure because it would be simply an automation of the Gimp method to all layer under in the stack, so maybe not so hard to do if the Gimp method is also done.




Offline maxy

  • Moderator
  • Hero Member
  • *****
  • Posts: 559
  • Karma: +9/-1
Re: Merging weird behavior
« Reply #6 on: January 16, 2012, 09:11:45 pm »
5. A proposition ? sure...  When performing a merge down layer, if the layer has a blending mode,
and so create an ambiguous situation , the merging should propose ( a modal dialog ) to :
- Affect only layer under (3, as the way of Gimp)
- Affect all layers ; a new choice described on picture (5). The user can keep the bunny and the background this way with the "layer blending mode" properly merged.
Deevad, obviously I will mention now that the alternative would be to use the same math that I used for brush modes in my blend_modes4 branch (see other threads here, or the blend_modes4.ogv video). That is, you don't affect the layers below, and the merged layer is in normal mode. The merged layer is made more opaque were needed, cloning from the layers below, such that the composited image doesn't change.

Offline David Revoy

  • Mypaint Dev
  • Full Member
  • *****
  • Posts: 207
  • Karma: +13/-0
    • portfolio
Re: Merging weird behavior
« Reply #7 on: January 17, 2012, 09:57:40 am »
Deevad, obviously I will mention now that the alternative would be to use the same math that I used for brush modes in my blend_modes4 branch (see other threads here, or the blend_modes4.ogv video). That is, you don't affect the layers below, and the merged layer is in normal mode. The merged layer is made more opaque were needed, cloning from the layers below, such that the composited image doesn't change.
Ohhh wow.  :o
And I'm in love for this of course :) I couldn't imagine this also to be possible with merging layer with a blending mode.
This is definitely the best solution I saw.

Offline rublev1360

  • Newbie
  • *
  • Posts: 31
  • Karma: +2/-0
    • andreabianco.eu
Re: Merging weird behavior
« Reply #8 on: January 17, 2012, 11:40:51 am »
@maxy: impressive, I'm in love with that too :)

I think it would be good to include Gimp's behavior as default for merging down layers, and then a better way like this one as an option in the preferences. This would also give a chance of a more coherent workflow for users that are working MyPaint & Gimp together :)

I created two fast videos showing how Painter and ArtRage manage this task, just in case.

Offline maxy

  • Moderator
  • Hero Member
  • *****
  • Posts: 559
  • Karma: +9/-1
Re: Merging weird behavior
« Reply #9 on: March 04, 2012, 08:32:33 pm »
I implemented it in a experimental branch, in case someone wants to test: http://gitorious.org/~maxy/mypaint/maxy-experimental/commits/rgbu

Here is a demo video: http://maxy.homeip.net/misc/convert-layer-to-normal-mode-rgbu.ogv

Offline maxy

  • Moderator
  • Hero Member
  • *****
  • Posts: 559
  • Karma: +9/-1
Re: Merging weird behavior
« Reply #10 on: March 04, 2012, 08:46:59 pm »
I think it would be good to include Gimp's behavior as default for merging down layers, and then a better way like this one as an option in the preferences. This would also give a chance of a more coherent workflow for users that are working MyPaint & Gimp together :)

I think that's mainly because you have gotten used to it for very long. If we want to support the GIMP behaviour in addition, then it certainly should not be the default - it's not what a user without strong CG background expects or desires. The only drawback I see in that branch now is the ugly alpha channel, but IMO that's better than to keep around clutter layers just because they don't merge down correctly.

Offline David Revoy

  • Mypaint Dev
  • Full Member
  • *****
  • Posts: 207
  • Karma: +13/-0
    • portfolio
Re: Merging weird behavior
« Reply #11 on: March 05, 2012, 07:44:45 am »
I implemented it in a experimental branch, in case someone wants to test: http://gitorious.org/~maxy/mypaint/maxy-experimental/commits/rgbu

Here is a demo video: http://maxy.homeip.net/misc/convert-layer-to-normal-mode-rgbu.ogv

Very impressive feature Maxy.

Offline achadwick

  • Administrator
  • Full Member
  • *****
  • Posts: 245
  • Karma: +9/-0
  • MyPaint developer and Mypaint-testing PPA owner
Re: Merging weird behavior
« Reply #12 on: March 05, 2012, 03:02:28 pm »
I think that's mainly because you have gotten used to it for very long. If we want to support the GIMP behaviour in addition, then it certainly should not be the default - it's not what a user without strong CG background expects or desires. The only drawback I see in that branch now is the ugly alpha channel, but IMO that's better than to keep around clutter layers just because they don't merge down correctly.

I'm really liking this "multi-merge" idea, and Deevad's explanation is brilliant! IMO, the best and simplest way of exposing this to users would be as a "Merge Visible Layers" command, junking the existing Merge Down implementation and ignoring the unhelpful GIMP implementation of "Merge Down" entirely. I'd like my hypothetical "Merge Visible Layers" to work like this:

  • I'll sometimes want to pick up the background texture, and sometimes I won't. But I always want what I see on the screen to be a preview of what I get.
  • Therefore, make the textured background a layer in the Layers list, but one without a lock button and which cannot be worked on. Its visibility can be turned off and on, and if it is made invisible, it is displayed as a chequered background, but crucially does not count as that for the purposes of compositing.
  • If the textured bg is made invisible, it counts as alpha=0 pure black for the purposes of compositing. Or grey, or white. This may mean that some effects layers now don't have any effect. That's fine! After all, the idea is that the outcome of this process is that you get what you see.

My hypothetical "Merge Visible Layers" command would work like this, in terms of what's currently on Maxy's rgbu branch:

Code: [Select]
# pseudocode, baby!
for each visible layer A in the stack, in ascending order:
    select A;
    perform the "Convert Layer to Normal" operation on A,
        picking up either the textured background
        or alpha=0 nothingness as above;
    if there is a visible layer B beneath A:
        if B is not the texture background:
             "Merge Down" onto B;
    continue with the next visible layer above A.

I think that this would be the easiest-to-understand way we could package this for users. Would there be any need for a separate "Convert Layer to Normal" command if we present this as "Merge Visible Layers"?

Offline savageorange

  • Full Member
  • ***
  • Posts: 158
  • Karma: +9/-0
Re: Merging weird behavior
« Reply #13 on: March 05, 2012, 09:30:54 pm »
.. yes. (I have plenty of use for the convert command -- not only for preparing for merge.; no particular use for the Merge Visible, although that could change)

Offline achadwick

  • Administrator
  • Full Member
  • *****
  • Posts: 245
  • Karma: +9/-0
  • MyPaint developer and Mypaint-testing PPA owner
Re: Merging weird behavior
« Reply #14 on: March 05, 2012, 11:05:19 pm »
.. yes. (I have plenty of use for the convert command -- not only for preparing for merge.; no particular use for the Merge Visible, although that could change)

Well, it'd ultimately contain it, assuming this idea works. Fingers crossed.