Category: AfterEffects

How To Compile PiPL Resource In Visual Studio

GOAL

Today’s goal is to compile PiPL resource with custom build step in Visual Studio while Adobe plug-in development.

If you’d like to know about PiPL resource, see the appendix “What is PiPL resource?” in this article.

Environment

Windows10
Visual Studio 2015 Community
After Effects CC2019
After Effects 17.1.2 Win SDK

Method

1. Open solution file with Visual Studio

Make sure .r is added to the current solution file.

2. Open properties of PiPL resource file

Right-click the PiPL resource file .r and click “Properties”.

3. Set Item Type

Select “Custom Build Tool” in Item Type list.

(more…)

How To Build Sample Projects In After Effects SDK

GOAL

Today’s goal is to build sample projects in AE SDK.

Environment

Windows10
Visual Studio 2015 Community
After Effects CC2019
After Effects 17.1.2 Win SDK

Method

Download AE SDK

Access Adobe Developer Console (https://console.adobe.io/servicesandapis) and sign in with your Adobe user account.

Select After Effects “View downloads” and download After Effects Plug-in SDK. Then extract downloaded folder.

Build single plug-in

Select project

Open AfterEffectsSDK\Examples and select a project that you’d like to build. For example, “AfterEffectsSDK\Examples\AEGP\Artie” is one of the project directory. See the list of sample project in Sample Project Descriptions.

Open solution file

Open the solution file (for example, AfterEffectsSDK\Examples\AEGP\Artie\Win\Artie.sln) with Visual Studio 2015(v140). While you can use later versions, you should upgrade the project from v140 to using version.

(more…)

How to adapt your AE plugin for 8, 16, 32bit color

Hello. In previous article “First After Effects Effects-Plug-In Development“, I implemented a simple After Effects plugin. This article is supplement to it.

What is color (bit) depth

Colors are represented as numbers in computers. In RGB color system, color is represented (R, G, B) such as (100, 120, 144) or (0.5, 0.2, 0.7). In gray scale mode, color is represented as one value such as a integer form 0 to 255 or float from 0 to 1.0.

Color bit depth is resolution of each values. When the color is 8 bpc(bit per channel) each channel, R, G, B and A is expressed using 8bit. When the color is 8bpp(bit per pixel), each pixel color is expressed using 8bit, 3bit for R and G and 2 bit for B.

The following is the image of the difference between 4bit color (4bpc) and 8bit color (8bpc).

Difference 4bpc and 8bpc

Color bit depth in After Effects

In After Effects, you can use 8-bpc, 16-bpc, or 32-bpc color. (check “Color depth and high dynamic range color” for details)

8bit: 0-255 for each channel (256 gradation)
16bit: 0-32767 for each channel (32,768 gradation)
32bit: The range represented by a 32-bit float for each channel (4,294,967,296 gradation)

The color is represented by integer in 8bpc and 16bpc color. Float value is clamped into [0.0 – 1.0] and convert to [0-255] or [0-32767] in 8bpc and 16bpc.

Color bit depth in AE plug-in development

When you develop After Effects plug-in, you should consider all color 8/16/32bit.

8bit

8ARGB color is stored using PF_Pixel8

//this is from AE_Effect.h

// Basic pixel defn's
typedef struct {
	A_u_char	alpha, red, green, blue;
} PF_Pixel;

typedef PF_Pixel		PF_Pixel8;
typedef PF_Pixel		PF_UnionablePixel;

16bit

//this is from AE_Effect.h

typedef struct {
	#ifdef PF_PIXEL16_RENAME_COMPONENTS
		// this style is useful for debugging code converted from 8 bit
		A_u_short		alphaSu, redSu, greenSu, blueSu;
	#else
		A_u_short		alpha, red, green, blue;
	#endif
} PF_Pixel16;

32bit

32bit color is supported in CS6 (11.0) and more.

//this is from AE_Effect.h

typedef A_FpShort			PF_FpShort;
typedef A_FpLong			PF_FpLong;
typedef struct {
	PF_FpShort				alpha, red, green, blue;
} PF_PixelFloat, PF_Pixel32;

And as you can see below, each channel is char type (8bit) in PF_Pixel or PF_Pixel8, unsigned short type (16bit) in PF_Pixel16 and float type (32bit) in PF_PixelFloat or PF_Pixel32.

//picked up from A.h

typedef char			A_char;
typedef float			A_FpShort;
typedef unsigned short	A_u_short;

Tips of color depth handling in AE plug-in development

Global setup

To make your effect handle 16-bpc color, set out_data->out_flags PF_OutFlag_DEEP_COLOR_AWARE.

And to make your effect supports 32-bpc colors, set out_data->out_flags2 PF_OutFlag2_FLOAT_COLOR_AWARE and PF_OutFlag2_SUPPORTS_SMART_RENDER.

Set pixel format

Before CS6, PF_CHECKOUT_PARAM() only returned 8-bit ARGB buffers, regardless of the pixel format currently being used for rendering. Starting in CS6, an effect can opt in to get frames in the same format as the render request, whether it is 32-bit float, YUV, etc.

from  After Effects SDK Guide

Render

In Render function, the iterate function scan input flame and calculate output frame as pixel to pixel operation, pixel function. The iterate functions is selected separately depending on the PrPixelFormat of destination pixels. Check After Effects SDK Guide “Progress During Iteration” for details.

Smart Render

In Smart Render function, pixel functions is selected separately depending on the PrPixelFormat like Render function.

If you want to know about the difference between Render function and SmartRender function, check the article “The difference of Render and Smart Render in After Effects”. (sorry, it will come soon)

The conditional branch by switch sentence is following.

 switch (format) {
    case PF_PixelFormat_ARGB128:
        iterateFloatSuite ->iterate(
            in_data, 0, output_worldP->height, input_worldP, 
            NULL, (void*)infoP, FilterImage32, output_worldP);
        break;
    case PF_PixelFormat_ARGB64:
        iterate16Suite ->iterate(
            in_data, 0, output_worldP->height, input_worldP,
            NULL, (void*)infoP, FilterImage16, output_worldP);
        break;
    case PF_PixelFormat_ARGB32:
        iterate8Suite ->iterate(
            in_data, 0, output_worldP->height, input_worldP,
            NULL, (void*)infoP, FilterImage8, output_worldP);
        break;
     default:
         err = PF_Err_BAD_CALLBACK_PARAM;
         break;
}

Pixel functions

Prepare each functions for 8/16/32 bit separately. The follow example is in SDK_Noise.cpp.

The differences are following.

*inP and *outP

// 8bit
static PF_Err FilterImage8 (
    void *refcon, A_long xL, A_long yL, 
    PF_Pixel8 *inP, PF_Pixel8 *outP)

// 16bit
static PF_Err FilterImage16 (
    void *refcon, A_long xL, A_long yL, 
    PF_Pixel16 *inP, PF_Pixel16 *outP)

// 32bit
static PF_Err FilterImage32 (
    void *refcon, A_long xL, A_long yL, 
    PF_PixelFloat *inP, PF_PixelFloat *outP)

tempF

// 8bit   clamp in range [0, PF_MAX_CHAN8]
tempF = rand() % PF_MAX_CHAN8;
tempF *= (niP->valF / SLIDER_MAX);

// 16bit   clamp in range [0, PF_MAX_CHAN16]
tempF = rand() % PF_MAX_CHAN16;
tempF *= (niP->valF / SLIDER_MAX);

// 32bit    clamp in range [0, 1 = PF_MAX_CHAN16/PF_MAX_CHAN16]
tempF = (PF_FpShort)(rand() % PF_MAX_CHAN16);
tempF *= (PF_FpShort)(niP->valF / (SLIDER_MAX * PF_MAX_CHAN16));

channels of outP

// 8bit  clamp in the range [0, PF_MAX_CHAN8] 
outP->red = MIN(PF_MAX_CHAN8, inP->red + (A_u_char) tempF);

// 16bit  clamp in the range[0, PF_MAX_CHAN16] 
outP->red = MIN(PF_MAX_CHAN16, inP->red + (A_u_short) tempF);

// 32bit
outP->red = (inP->red + tempF);

The Environment for After Effects Plugin Development

GOAL

To set up the environment for After Effects plugin. To build sample plug-in and execute it.

Environment

Adobe CC2019 v16.0.0
Windows10 with environment for C++ compile
Microsoft Visual C++ 2017 ver15.8.4

Method

1 Download The After Effects SDKs

Access Adobe Web site and click “Get the SDKs” button to get After Effects SDKs. Select “After Effects Plug-in SDK” for your AE version.

2 Set build path as environment variable

Add environment variable “AE_PLUGIN_BUILD_DIR” and set output directory. If it doesn’t exist, the error “fatal error LNK1104: cannot open file ‘***.aex” will occur.

3 Build sample plug-in

Unzip “After+Effects+CC+16.0+Win+SDK.zip” and put the unzipped directory “AfterEffectsSDK” into appropriate location.

Open AfterEffectsSDK\Examples\BuildAll.sln then Visual studio will start.

Right Click on “SDK Noise” and Build it.

from Visual Studio

If the error “C2220: warning treated as error – no ‘object’ file generated” occurred, fix the project configuration not to treat warning as error.

from Visual Studio

4 Put the plugin into AE Directory

Put the “SDK_Noise.aex” file into “C:\Program Files\Adobe\Adobe After Effects CC 2019\Support Files” Directory.

5 Add plug-in Effect

Open AE project and click Effect>Sample Plug-ins>SDK_Noise

from Adobe AfterEffects

A noise effect is added to the video.

First After Effects Effects-Plug-In Development

GOAL

Develop my first After Effects Plugin, ColorChange effect.

Environment

Adobe CC2019 v16.0.0
Windows10 with environment for C++ compile
Microsoft Visual C++ 2017 ver15.8.4

Method

Environment setup

Set up the environment for After Effects plugin with reference to “The Environment for After Effects Plugin Development.”

Copy Template

Copy Skeleton template from “AfterEffectsSDK\Examples\Template\Skeleton” , with directory hierarchies of “Skeleton”, “Headers”, “Resources” and “Util” preserved .

Rename file names and replace “Skeleton” with “Color Change” in it. DON’T rename “SkeletonPiPL.rc” because binary .rc file is automatically generated from .r file (AE Plugin SDK Guide “PiPL Resources”).

EffectMain Function

Main function is defined in SkeletonPiPL.r as follows.

#ifdef AE_OS_WIN
	#ifdef AE_PROC_INTELx64
		CodeWin64X86 {"EffectMain"},
	#endif
#else
	#ifdef AE_OS_MAC
		CodeMacIntel64 {"EffectMain"},
	#endif
#endif

EffectMain is the Function accepts PF_Cmd cmd as a parameter and call functions using cmd as a selector such as “PF_Cmd_ABOUT”, “PF_Cmd_PARAMS_SETUP” and “PF_Cmd_RENDER”. Refer to “AE Plugin SDK Guide command selectors.”

PF_Err EffectMain(
	PF_Cmd			cmd,
	PF_InData		*in_data,
	PF_OutData		*out_data,
	PF_ParamDef		*params[],
	PF_LayerDef		*output,
	void			*extra)
{
  PF_Err err = PF_Err_NONE;
  try {
    switch (cmd) {
    case PF_Cmd_ABOUT:
      err = About(in_data,
		  out_data,
		  params,
		  output);
	break;
    case PF_Cmd_GLOBAL_SETUP:
      err = GlobalSetup(in_data,
                        out_data,
                        params,
                        output);
      break;
    case PF_Cmd_PARAMS_SETUP:
      err = ParamsSetup(in_data,
                        out_data,
                        params,
                        output);
      break;
    case PF_Cmd_RENDER:
      err = Render(in_data,
                   out_data,
                   params,
                   output);
      break;
    }
  }
  catch(PF_Err &thrown_err){
    err = thrown_err;
  }
  return err;
}

About Function

About is the function to display a dialog describing the plug-in. Change TableString in ColorChange_Stgrings.cpp as follows.

TableString g_strs[StrID_NUMTYPES] = {
	StrID_NONE,		"",
	StrID_Name,		"ColorChange",
	StrID_Description,	"Change comp color to specified color",
	StrID_Gain_Param_Name,	"Gain",
	StrID_Color_Param_Name,	"Color",
};
Open dialog
The message is displayed

GlobalSetup

Now GlobalSetup function doesn’t need any change.

ParamsSetup

ParamsSetup is the function to setup UI, describe your parameters and register them.

I renamed “SKELETON” to “COLORCHANGE”, delete “GAIN”parameter and added “LEVEL” parameter. Then reset values as follows in ColorChange.h.

/* Parameter defaults */

#define	COLORCHANGE_LEVEL_MIN	0
#define	COLORCHANGE_LEVEL_MAX	100
#define	COLORCHANGE_LEVEL_DFLT	50
enum {
	COLORCHANGE_INPUT = 0,
	COLOECHANGE_LEVEL,
	COLORCHANGE_COLOR,
	COLORCHANGE_NUM_PARAMS
};
enum {
	LEVEL_DISK_ID = 1,
	COLOR_DISK_ID,
};

Accordingly, rename constant names in ColorChange.cpp.

Render

Render function is the function to render the effect into the output, based on the input and parameters.

GainInfo

GainInfo is the structure to handle parameter “GAIN”. Create new structure for passing parameter data “level” and “color” in ColorChange.h.

typedef struct ParamInfo {
	PF_FpLong level;
	PF_Pixel color;
	PF_Pixel16 color16;
} PramInfo, *PramInfoP, **PramInfoH;
//GainInfo          giP;
//AEFX_CLR_STRUCT(giP);
ParamInfo            paramDataP;
AEFX_CLR_STRUCT(paramDataP);
A_long              linesL  = 0;

linesL      = output->extent_hint.bottom - output->extent_hint.top;
paramDataP.level = params[COLOECHANGE_LEVEL]->u.fs_d.value;
paramDataP.color = params[COLORCHANGE_COLOR]->u.cd.value;
iterate

The iterate function scan input flame and calculate output frame as pixel to pixel operation, pixel function. In this case, the pixel function is “MySimpleGainFunc16” or “MySimpleGainFunc8”. Rename and change them into “MyColorChangeFunc16” and “MyColorChangeFunc8”.

if (PF_WORLD_IS_DEEP(output)){
    paramDataP.color16.red = CONVERT8TO16(paramDataP.color.red);
    paramDataP.color16.green = CONVERT8TO16(paramDataP.color.green);
    paramDataP.color16.blue = CONVERT8TO16(paramDataP.color.blue);
    paramDataP.color16.alpha = CONVERT8TO16(paramDataP.color.alpha);
    ERR(suites.Iterate16Suite1()->iterate(
        in_data,
        0,                                          // progress base
        linesL,                                 // progress final
        &params[COLORCHANGE_INPUT]->u.ld,	// src 
        NULL,			                 // area - null for all pixels
        (void*)&paramDataP,      // refcon - your custom data pointer
        MyColorChangeFunc16, // pixel function pointer
        output));
}else {
    ERR(suites.Iterate8Suite1()->iterate(
        in_data,
	0,                            // progress base
	linesL,                   // progress final
	&params[COLORCHANGE_INPUT]->u.ld,	// src 
	NULL,                   // area - null for all pixels
	(void*)&paramDataP,       // refcon - your custom data pointer
	MyColorChangeFunc8,     // pixel function pointer
	output));	
}
MyColorChangeFunc

I changed the pixel function as follows.

static PF_Err MyColorChangeFunc8 (
	void		*refcon, 
	A_long		xL, 
	A_long		yL, 
	PF_Pixel8	*inP, 
	PF_Pixel8	*outP)
{
	PF_Err err = PF_Err_NONE;
	ParamInfo *paramDataP = reinterpret_cast<ParamInfo*>(refcon);
	PF_FpLong levelF = 0;
	float red_diff, green_diff, blue_diff;
			
	if (paramDataP){
		levelF = paramDataP->level/100.0;
		red_diff = (paramDataP->color.red-inP->red)*levelF;
		green_diff = (paramDataP->color.green-inP->green)*levelF;
		blue_diff = (paramDataP->color.blue-inP->blue)*levelF;

		outP->alpha		=	inP->alpha;
		outP->red = MIN(inP->red + red_diff, PF_MAX_CHAN8);
		outP->green = MIN(inP->green +green_diff, PF_MAX_CHAN8);
		outP->blue = MIN(inP->blue + blue_diff, PF_MAX_CHAN8);
	}
	return err;
}
static PF_Err MyColorChangeFunc16 (
	void		*refcon, 
	A_long		xL, 
	A_long		yL, 
	PF_Pixel16	*inP, 
	PF_Pixel16	*outP)
{
	PF_Err err = PF_Err_NONE;
	ParamInfo	*paramDataP	= reinterpret_cast<ParamInfo*>(refcon);
	PF_FpLong	levelF	= 0;
	float red_diff, green_diff, blue_diff;

	if (paramDataP) {
		levelF = paramDataP->level / 100.0;
		red_diff = (paramDataP->color16.red - inP->red)*levelF;
		green_diff = (paramDataP->color16.green - inP->green)*levelF;
		blue_diff = (paramDataP->color16.blue - inP->blue)*levelF;

		outP->alpha = inP->alpha;
		outP->red = MIN(inP->red + red_diff, PF_MAX_CHAN16);
		outP->green = MIN(inP->green + green_diff, PF_MAX_CHAN16);
		outP->blue = MIN(inP->blue + blue_diff, PF_MAX_CHAN16);
	}
	return err;
}

Build and Install

Build solution and put generated .aex file into Adobe AE directory such as “C:\Program Files\Adobe\Adobe After Effects CC 2019\Support Files.”

Appenedix

2020/09/06 Changed source codes to support 16bpc color. Thank you for comments and pointing it out.
Please take a look at the article “How to adapt your AE plugin for 8, 16, 32bit color” as well.

What is color manager

GOAL

To understand color space and color manager.

What is color manager?

Color manager (also called “color management system” or “management solution”) is an application to change the color space of image and video.

In VFX production, many color spaces are used according to the tools such as digital cinema camera, rendered CG, photos and matte paint. it is necessary to unify or convert the color space. Color space conversion is an important process in VFX production.

What is color space?

Color space is a specific organization of colors in which colors are represented as coordinates.

Color model

Color model is an abstract mathematical model describing colors with some channels ( e.g. RGB, CMYK). Although color model is not the same thing as color space, some color system such as Adobe RGB and sRGB are based on color model.

Representative color space

Color spaces in Adobe Photoshop Color Picker
Comparison of some RGB and CMYK colour gamut on a CIE 1931 xy chromaticity diagram, based on http://commons.wikimedia.org/wiki/File:CIE1931xy_blank.svg and data from Blatner and Fraser’s “Real World Photoshop CS”, p179: http://canopuscomputing.com.au/gallery2/d/8631-1/Gamuts_comparison-B_F.jpg . ( CC BY-SA 3.0 )

sRGB

sRGB (standard Red Green Blue) is an RGB color space to use mainly on monitors, printers, and the Internet. The color gamut that can be represented in this method is the color triangle defined by three primary colors, Red, Green and Blue.

CIE 1931 xy chromaticity diagram showing the gamut of the sRGB color space and location of the primaries.( CC BY-SA 3.0)
* This image is colored in sRGB space, so the colors outside of the triangle are interpolated.

Adobe RGB

Adobe RGB is a color space definition proposed by Adobe Systems. It has a wider RGB color reproduction range than sRGB (especially green), designed to cover most of the colors achievable with CMYK color printers. It encompasses about 50% of the visible colors specified in the CIELAB color space.

The CIE 1931 xy chromaticity diagram showing the primaries of the Adobe RGB (1998) color space. The CIE Standard Illuminant D65 white point is shown in the center.( CC BY-SA 3.0)

NTSC (BT.601)

NTSC Color Space is designed for television. It features a color gamut that is much wider than sRGB.While NTSC is not used in modern displays, it is commonly used to compare and specify color gamut.
This is also one of the major television formats in the world including PAL, SECAM.

Rec.2020

ITU-R Recommendation BT.2020 aka Rec.2020 or BT.2020 defines various aspects of ultra-high-definition television (UHDTV) with standard dynamic range (SDR) and wide color gamut (WCG).
It defines picture resolutions, frame rates with progressive scan, bit depths, color primaries, RGB and luma-chroma color representations, chroma subsamplings, and an opto-electronic transfer function. full HD and HDR are not supported.

Rec.2100

Rec.2100 is upward compatible with Rec.2020. ITU-R Recommendation BT.2100 aka Rec.2100 or BT.2100 is an international standard for specifications that must be met by devices that handle full HD(2K), 4K and 8K resolutions. It was established by the International Telecommunication Union Wireless Communication Sector (ITU-R).

CMYK(or just CMY)

CMYK uses subtractive color mixing used in the printing process.
CMYK corresponds to Ink colors, Cyan, Magenta, Yellow and Black.

HSV 

HSV(HueSaturationValue) is used for painting or color sample on computers. Painting artists use it because it is more natural and intuitive to consider colors in terms of hue, color and saturation than additive mixing or subtractive mixing. HSV is a transformation of an RGB color space. HSV is also called HSB (hue, saturation, brightness).

HSL

HSL (hue, saturation, lightness / luminance) is quite similar to HSV, with “lightness” replacing “brightness”. The difference is a value calculation method. HSV uses a hexagonal pyramid model in which the brightness of pure color is equal to the brightness of white, while HLS uses a bi-hexagonal pyramid model in which the brightness of pure color is 50% of the brightness of white.(Please refer this image). It is also called HSI (hue, saturation, intensity).

LMS

LMS color place is based on three kinds of cone cells that human eye with normal vision has. These cone cells sense light and have peaks of spectral sensitivity in Short wavelength, Middle wavelength and Long wavelength.
The three parameters corresponding to levels of stimulus of the three kinds of cone cells(L, M, S) describe any human color sensation. So LMS color space can contain all visible colors.
However LMS is not objective representation of colors because parameters L, M and S is different between people and emvironment.

CIE 1931 color spaces

CIE (Commission internationale de l’éclairage, the International Commission on Illumination) is the organization that creates international standards related to light and color.
CIE 1931 color spaces is the first defined quantitative relations between distributions of wavelengths in the electromagnetic visible spectrum, and physiologically perceived colors.

The CIE XYZ is remap of the color space in which the tristimulus values ​​are conceptualized as amounts of three primary colors.These primary colors are unvisible for human. The CIE XYZ color space is designed so that Y component corresponds to luminance.
CIE XYZ can contain all visible colors.While RGB can’t represent some of visible colors without using negative value, CIE XYZ can contain all visible colors in positive quadrant.

Why is color management needed?

©2020 Nako

When you transfer exported video to another media, color gamut remapping, gamma correction, setting of white chromaticity (white point) are required and so on.

What is ACES?

ACES(Academy Color Encoding System) is a color image encoding system created under the auspices of the Academy of Motion Picture Arts and Sciences. The system defines its own primary color that encompass the visible spectral locus as defined by the CIE xyY specification.

When you use ACES in Video editing software such as Adobe AfterEffects , Maya and Nuke, color manager is needed. Color manager is usually provided as a plugin or built-in function.

Representative Color manager

OpenColorIO

OpenColorIO (OCIO) is a complete color management solution geared towards motion picture production with an emphasis on visual effects and computer animation.

OpenColorIO official web site( https://opencolorio.org/ )

OCIO is compatible with ACES and is LUT-format agnostic, supporting many popular formats.

SynColor

SynColor is the Autodesk Color Management component.

Adobe Color Management Module(CMM)

CMM is color manager for Adobe software such as Photoshop.