ISO-14496-16-AMD-2-2009.pdf

Reference number ISO/IEC 14496-16:2006/Amd.2:2009(E) © ISO/IEC 2009 INTERNATIONAL STANDARD ISO/IEC 14496-16 Second edition 2006-12-15 AMENDMENT 2 2009-02-15 Information technology Coding of audio-visual objects Part 16: Animation Framework eXtension (AFX) AMENDMENT 2: Frame-based Animated Mesh Compression (FAMC) Technologies de l'information Codage des objets audiovisuels Partie 16: Extension du cadre d'animation (AFX) AMENDEMENT 2: Compression trame par trame de maillage animé (FAMC) ISO/IEC 14496-16:2006/Amd.2:2009(E) PDF disclaimer This PDF file may contain embedded typefaces. In accordance with Adobe's licensing policy, this file may be printed or viewed but shall not be edited unless the typefaces which are embedded are licensed to and installed on the computer performing the editing. In downloading this file, parties accept therein the responsibility of not infringing Adobe's licensing policy. The ISO Central Secretariat accepts no liability in this area. Adobe is a trademark of Adobe Systems Incorporated. Details of the software products used to create this PDF file can be found in the General Info relative to the file; the PDF-creation parameters were optimized for printing. Every care has been taken to ensure that the file is suitable for use by ISO member bodies. In the unlikely event that a problem relating to it is found, please inform the Central Secretariat at the address given below. COPYRIGHT PROTECTED DOCUMENT © ISO/IEC 2009 All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic or mechanical, including photocopying and microfilm, without permission in writing from either ISO at the address below or ISO's member body in the country of the requester. ISO copyright office Case postale 56 CH-1211 Geneva 20 Tel. + 41 22 749 01 11 Fax + 41 22 749 09 47 E-mail copyrightiso.org Web www.iso.org Published in Switzerland ii © ISO/IEC 2009 All rights reserved -,-,- ISO/IEC 14496-16:2006/Amd.2:2009(E) © ISO/IEC 2009 All rights reserved iii Foreword ISO (the International Organization for Standardization) and IEC (the International Electrotechnical Commission) form the specialized system for worldwide standardization. National bodies that are members of ISO or IEC participate in the development of International Standards through technical committees established by the respective organization to deal with particular fields of technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work. In the field of information technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 2. The main task of the joint technical committee is to prepare International Standards. Draft International Standards adopted by the joint technical committee are circulated to national bodies for voting. Publication as an International Standard requires approval by at least 75 % of the national bodies casting a vote. Attention is drawn to the possibility that some of the elements of this document may be the subject of patent rights. ISO and IEC shall not be held responsible for identifying any or all such patent rights. Amendment 2 to ISO/IEC 14496-16:2006 was prepared by Joint Technical Committee ISO/IEC JTC 1, Information technology, Subcommittee SC 29, Coding of audio, picture, multimedia and hypermedia information. ISO/IEC 14496-16 introduced several animation models as methods of deforming a mesh. Amendment 2 to ISO/IEC 14496-16:2006 deals with decoding animation data (mainly vertex coordinates and attributes, temporally updated) independently of a mesh deformation model. -,-,- ISO/IEC 14496-16:2006/Amd.2:2009(E) © ISO/IEC 2009 All rights reserved 1 Information technology Coding of audio-visual objects Part 16: Animation Framework eXtension (AFX) AMENDMENT 2: Frame-based Animated Mesh Compression (FAMC) After 5.9, add the following new subclause: 5.10 Frame-based Animated Mesh Compression (FAMC) stream 5.10.1 Overview FAMC is a tool to compress an animated mesh by encoding on a time basis the attributes (position, normals ) of vertices composing the mesh. FAMC is independent on the manner how animation is obtained (deformation or rigid motion). The data in a FAMC stream is structured in segments of several frames. Each segment can be decoded individually. Within a segment, a temporal prediction model, called skinning, is represented. The model is used for motion compensation inside the segment. The FAMC bitstream structure is illustrated in Figure AMD2.1. Figure AMD2.1 FAMC bitstream structure. Each decoded animation frame updates the geometry and possibly the attributes (or only the attributes) of the 3D graphic object that FAMC is referred to. An animation segment contains two types of information: ISO/IEC 14496-16:2006/Amd.2:2009(E) 2 © ISO/IEC 2009 All rights reserved A header buffer indicating general information about the animation segment (number of frames, attributes to be updated). A data buffer containing: o The skinning model used for 3D motion compensation consists in a segmentation of the 3D mesh into clusters and is specified by: ? the partition information, i.e. the segmentation of the 3D object vertices into clusters, ? a set of animation weights connecting each vertex of the 3D object to each cluster and ? the motion data described in terms of a 3D affine transform for each cluster and for each animation frame. o The residual errors per vertex equal with the difference between the real value and the one predicted by the skinned motion compensation model, that are encoded with one of the following combination ? a Discrete Cosine Transform performed on the entire animation segment (referred in this document as DCT) ? an integer-to-integer Wavelet Transform performed on the entire animation segment (referred in this document as Lift). ? Layer based decomposition (referred in this document as LD) ? DCT followed by LD ? Lift followed by LD The prediction residual errors may correspond to geometric and/or attribute data. Figure AMD2.2 illustrates the FAMC decoding process. ISO/IEC 14496-16:2006/Amd.2:2009(E) © ISO/IEC 2009 All rights reserved 3 Figure AMD2.2 FAMC decoding process. The following sections describe in detail the structure of the FAMC stream. 5.10.2 FAMC inclusion in the scene graph FAMC is associated with an IndexedFaceSet by using the BitWrapper mechanism with value of field type equals to 2. 5.10.3 FAMC class 5.10.3.1 Syntax class FAMCAnimation do FAMCAnimationSegment animationSegment; bit(32)* next; while (next=FAMCAnimationSegmentStartCode); 5.10.3.2 Semantics FAMCAnimationSegmentStartCode: a constant that indicates the beginning of a FAMC animation segment. FAMCAnimationSegmentStartCode = 00 00 01 F0. ISO/IEC 14496-16:2006/Amd.2:2009(E) 4 © ISO/IEC 2009 All rights reserved 5.10.4 FAMCAnimationSegment class 5.10.4.1 Syntax class FAMCAnimationSegment FAMCAnimationSegmentHeader header; FAMCAnimationSegmentData data; 5.10.4.2 Semantics FAMCAnimationSegmentHeader: contains the header buffer. FAMCAnimationSegmentData: contains the data buffer. 5.10.5 FAMCAnimationSegmentHeader class 5.10.5.1 Syntax class FAMCAnimationSegmentHeader unsigned int (32) startCode; unsigned int (8) staticMeshDecodingType unsigned int (32) animationSegmentSize bit(4) animatedFields; bit(3) transformType; bit(1) interpolationNeeded; bit(2) normalsPredictionStrategy; bit(2) colorsPredictionStrategy; bit(4) otherAttributesPredictionStrategy; unsigned int (32) numberOfFrames; for(int f = 0; f largestAllowedPredDim ) predDimRes *= -1; else if( previousDimc - predDimRes = 0 ) if( cabac.biari_decode_symbol_eq_prob( cabac._dep ) ) predDimRes *= -1; predDimvc = predDimRes + previousDimc; previousDimc = predDimvc; else -,-,- ISO/IEC 14496-16:2006/Amd.2:2009(E) © ISO/IEC 2009 All rights reserved 11 predDimvc = 0; / end the arithmetic coding engine cabac.biari_decode_final( cabac._dep ); 5.10.10.2 Semantics delta: reciprocal value of the quantization step size. sigMapVF3: array of 3 * V * F bits, indicating the non-zero predicted spectral coefficients of x-, y- and z- component. EGk: order of the Exp-Golomb binarization. cutOff: number of CABAC context models for the unary part of the concatenated unary/ k-th order Exp- Golomb binarization. absValuesVF3: array of 3 * V * F integer values, indicating the absolute values of the predicted spectral coefficients of x-, y- and z-component. valuesVF3: array of 3 * V * F integer values, indicating the values of the predicted spectral coefficients including signs of x-, y- and z-component. predV3: an array indicating the index of the coefficient used for prediction of the current coefficient of x-, y- and z-component. predDimV3: the number of the samples that are used for predicting of x-, y- and z-component. The FAMCCABACDecoder class decodes a (V x F) array of three dimensional vectors of integer values. In order to obtain the original values the decoder should inverse the prediction stage as described in the following pseudo-code: / Inverse prediction for( int v = 1; v = 0; pb-) int bitOfBitPlane = cabac.biari_decode_symbol_eq_prob(cabac._dep); vertexIndex += (bitOfBitPlane * (1= 0; bp-) bool bitOfClusterIndex = cabac.biari_decode_symbol_eq_prob(cabac._dep); / decoding weights for (int bp = numberOfQuantizationBits -1; bp= 0; bp-) for(int v = 0; v =1; -layer) FAMCSimplificationModeDecoder myFAMCSimplificationModeDecoder (numberOfVerticesInLayerlayer); 5.10.17.2 Semantics compressedPartitionBufferSize: a 32 bit unsigned integer indicating the compressed stream size of the partition in layers. compressedSimplificationBufferSize: a 32 bit unsigned integer indicating the compressed stream size. The FAMCLayeredDecompositionDecoder class decodes a sequence of simplification operations. Each simplification operation is represented as a couple (vertexIndex, mode), both values are unsigned integers. First, the FAMCVertexInfoDecoder class decodes an array (noted partition) of integers of size V. Each element partitionv of the array is in the range 0,L-1 and indicats the assignement of vertex v to layer partitionv . The array of integers numberOfVerticesInLayer is obtained from the decoded array partition. The derivation process is illustrated with the following pseudo code: ISO/IEC 14496-16:2006/Amd.2:2009(E) © ISO/IEC 2009 All rights reserved 23 int numberOfVerticesInLayerL; for (v=0; v vvsop(L); for (int v=0; v=1; -layer) for (int c=0; c= 0; pb-) for (int v = 0; v isDecoded(numberOfVertices, false); for (l=0; l0 decoded data is used additionally for derivation of ldl). When computing ldnumberOfDecomposedLayers-1,ld0 all three arrays describing connectivity does not change their sizes, only values of their elements may change. In the following is described how layers of a layered decomposition are derived by successively processing and manipulating mesh connectivity starting with the initial connectivity. K.2 Deriving layers ldl, l0 using decoded data and connectivity (layeredDecompositionIsEncoded equals 1) From the data decoded by the FAMCLayeredDecompositionDecoder class, simplification operations vvsoplc (with l= numberOfDecomposedLayers-1,1 and c=0, numberOfVerticesInLayerl) are derived. Note that arrays vvsopl, l0 have the same sizes as arrays ldl,l0 will have after their derivation. Layers ldl are derived successively for l=numberOfDecomposedLayers-1,1. For this simplification operrations vvsoplc are applied for each l for c=0,numberOfVerticesInLayer. Each time before applying a simplification operation the follwing data is stored: in ldlc .to the vertex index to be removed vvsoplc.vertexIndex is stored, and in ldlc.from all vertex indices of the one-ring neighborhood of vertex vvsoplc.vertexIndex are stored. After deriving a layer ldl all its elements are stored in reverse order. The following pseudo code illustrates the derivation process of ld: for (int l=numberOfDecomposedLayer-1; l=1; -l) for (int c=0; c0 using only connectivity (default option, layeredDecompositionIsEncoded equals 0) The only difference of this derivation process compared to the derivation process described in previous section is that now simplification operation vvsop are not computed using decoded data and mesh connectivity, but they are computed from mesh connectivity only. In order to specify how layers ldl for l0 are computed it is sufficient to describe how simplification operations vvsop are derived from connectivity; the rest of the procedure is similar with the one specified in the previous section. Using initial connectivity as input, layers ldl are obtained in the order l = numberOfDecomposedLayers-1,1. Thereby, each layer ldl is derived in two stages: a) determination of a decomposition in patches patchl, b) derivation of simplification operations vvsopl using patchl and simplification. Simplified connectivity is applied for obtaining ldl-1. Simplified connectivity obtained after obtaining ld1 is used for computing ld0. This derivation process is described in K.4. In stage a) the mesh connectivity is decomposed in non-overlapping patches as illustrated in Figure K.1. All patches are described by an array of vertex indices patchlc with c=0,numberOfPatchesInLayerl. A patch patchlc consists of all faces containing vertex index patchlc. The number of these faces is called degree of vertex patchlc. Figure K.1 Decomposition of the mesh in patches. Gray shaded faces are part of patches, while white faces are not. The applied decomposition algorithm is a patch-based breath-first region-growing algorithm, which determines only patches with degree less or equal to 6. In the following the pseudo code of function decompose(patch) is given, which describes the decomposition algorithm: void decompose(int patch) int seed; int vCurrent; FIFO fifo; int localVertices; bool vertexTag1 = isVertexDeleted; bool vertexTag2 = isVertexDeleted; bool faceTag = isFaceDeleted; -,-,- ISO/IEC 14496-16:2006/Amd.2:2009(E) 32 © ISO/IEC 2009 All rights reserved while (seed=getNextSeed()!=-1) fifo.push(seed); while (!fifo.empty() vCurrent=fifo.front(); fifo.pop(); if (isFreePatch(vCurrent) int deg = degree(vCurrent); if (3=0) | (0=0) | (0 isVisited(numberOfVerticesInLayer0,false); for (int c=0; c 6