Asset Storage Unification

Status: Not started Goal: Migrate builder uploads from local Drive to BunnyCDN with WebP conversion, Asset DB rows, and full scope hierarchy (brand/campaign/group/creative).

Context

Two separate upload systems exist today:

Asset library (POST /asset-library/upload): uploads to BunnyCDN Storage, creates Asset DB row, scoped to brand/campaign
Builder uploads (POST /assets): uploads to local Drive disk, no DB row, flat general/ folder with no scope

McSneaky confirmed the delivery server does nothing special beyond serving static files, so all new uploads can go to BunnyCDN. Old assets keep working from the old domain.

Out of scope

Video uploads (separate task, involves Bunny Stream)
URL hierarchy refactor for navigation (separate doc: todos/url-hierarchy-refactor.md)
Migrating existing assets from Drive to BunnyCDN (old URLs keep working)

Architecture

CreativeAssetsController switches from Adonis Drive (local disk) to BunnyStorage (CDN). Accepts optional scope IDs from frontend, builds hierarchical storage paths, creates Asset DB rows. Frontend sends scope IDs from Vuex store via FileUploader. WebP conversion in AssetLibraryController already done separately.

Tech Stack: AdonisJS 5, sharp, BunnyStorage service, Japa test framework, sinon stubs, Vue 2 + Vuex

Repos:

Backend: /Users/nicolay/CavaiProduct/Application-Backend (branch: asset_library)
Frontend: /Users/nicolay/CavaiProduct/Application-Frontend (branch: asset_library)

Task 1: DB Migration -- add creative scope columns

Files:

Create: database/migrations/{timestamp}_add_creative_scope_to_assets.ts
[ ] Step 1: Create migration file

typescript

import BaseSchema from '@ioc:Adonis/Lucid/Schema'

export default class AddCreativeScopeToAssets extends BaseSchema {
  protected tableName = 'assets'

  public async up() {
    this.schema.alterTable(this.tableName, (table) => {
      table.string('creative_group_id').nullable()
      table.string('creative_id').nullable()
      table.index('creative_group_id')
      table.index('creative_id')
    })
  }

  public async down() {
    this.schema.alterTable(this.tableName, (table) => {
      table.dropIndex('creative_group_id')
      table.dropIndex('creative_id')
      table.dropColumn('creative_group_id')
      table.dropColumn('creative_id')
    })
  }
}

Generate the file with: node ace make:migration add_creative_scope_to_assets Then replace its contents with the above.

[ ] Step 2: Add columns to Asset model

File: app/Models/Asset.ts

Add after the campaignId column (around line 44):

typescript

@column()
public creativeGroupId: string | null

@column()
public creativeId: string | null

[ ] Step 3: Run migration

bash

node ace migration:run

[ ] Step 4: Update AssetFactory with new columns

File: database/factories/AssetFactory.ts

Add creativeGroupId: null and creativeId: null to the factory defaults.

Task 2: Update StoreAssetValidator

Files:

Modify: app/Validators/StoreAssetValidator.ts
[ ] Step 1: Add optional creative scope fields

Add to the schema:

typescript

creative_group_id: schema.string.optional([rules.bigInt()]),
creative_id: schema.string.optional([rules.bigInt()]),

Add messages:

typescript

'creative_group_id': 'Creative group ID must be a valid ID',
'creative_id': 'Creative ID must be a valid ID',

Task 3: Update AssetLibraryController.store() for new columns

Files:

Modify: app/Controllers/Http/AssetLibraryController.ts
[ ] Step 1: Accept and persist creative scope fields

In store(), the validator payload already includes the new optional fields from Task 2. Update the Asset.create() call to include them:

typescript

creativeGroupId: payload.creative_group_id ?? null,
creativeId: payload.creative_id ?? null,

[ ] Step 2: Build hierarchical storage path

Replace the storage path line:

typescript

// Old: const storagePath = `${brand.workspaceId}/${brand.id}/${cuid()}.${finalExt}`

const pathSegments = [brand.workspaceId, brand.id]
if (campaign) pathSegments.push(campaign.id)
if (payload.creative_group_id) pathSegments.push(payload.creative_group_id)
if (payload.creative_id) pathSegments.push(payload.creative_id)
pathSegments.push(`${cuid()}.${finalExt}`)

const storagePath = pathSegments.join('/')

Task 4: Migrate CreativeAssetsController to BunnyCDN

Files:

Modify: app/Controllers/Http/CreativeAssetsController.ts

This is the main change. The controller currently uploads to local Drive. We switch it to BunnyCDN, add WebP conversion, and optionally create Asset DB rows.

[ ] Step 1: Update imports

Replace the Drive/readFile imports and add BunnyCDN + Asset imports:

typescript

import { promises as fsPromises } from 'fs'
import sharp from 'sharp'
import Asset from 'App/Models/Asset'
import Brand from 'App/Models/Brand'
import Campaign from 'App/Models/Campaign'
import BunnyStorage from 'App/Services/BunnyStorage'
import AssetMetadataExtractor from 'App/Services/AssetMetadataExtractor'
import { cuid } from '@ioc:Adonis/Core/Helpers'
import { schema, rules } from '@ioc:Adonis/Core/Validator'
import { HttpContextContract } from '@ioc:Adonis/Core/HttpContext'

Remove the Drive and CloudFlare imports (CloudFlare is only used by getUploadUrl which we keep unchanged). Remove readFile from fs/promises (we use fsPromises from fs instead). Keep Env if getUploadUrl still needs it.

[ ] Step 2: Rewrite the store() method

Replace the entire store() method:

typescript

public async store({ request, response, auth }: HttpContextContract) {
  const ALLOWED_IMAGES = ['jpg', 'jpeg', 'png', 'gif', 'webp']
  const ALLOWED_FONTS = ['woff', 'woff2', 'ttf', 'otf']
  const ALLOWED_VR_FILES = ['glb', 'gltf', 'usdz']
  const WEBP_CONVERTIBLE = ['jpg', 'jpeg', 'png']

  const payload = await request.validate({
    schema: schema.create({
      resize_width: schema.number.optional(),
      resize_height: schema.number.optional(),
      brand_id: schema.string.optional([rules.bigInt()]),
      campaign_id: schema.string.optional([rules.bigInt()]),
      creative_group_id: schema.string.optional([rules.bigInt()]),
      creative_id: schema.string.optional([rules.bigInt()]),
      file: schema.file({
        size: '10mb',
        extnames: [...ALLOWED_IMAGES, ...ALLOWED_FONTS, ...ALLOWED_VR_FILES],
      }),
    }),
  })

  const uploadedFile = payload.file
  const originalExt = uploadedFile.extname!
  const shouldConvertToWebp = WEBP_CONVERTIBLE.includes(originalExt.toLowerCase())

  // Classify file type
  let fileType = ''
  if (ALLOWED_IMAGES.includes(originalExt)) fileType = 'image'
  else if (ALLOWED_FONTS.includes(originalExt)) fileType = 'font'
  else if (ALLOWED_VR_FILES.includes(originalExt)) fileType = 'vr'

  // Read and optionally convert to WebP
  const originalBuffer = await fsPromises.readFile(uploadedFile.tmpPath!)

  let uploadBuffer: Buffer
  let finalExt: string
  let fileSize: number
  let width: number | undefined
  let height: number | undefined

  if (shouldConvertToWebp) {
    const result = await sharp(originalBuffer)
      .webp({ quality: 100 })
      .toBuffer({ resolveWithObject: true })

    uploadBuffer = result.data
    finalExt = 'webp'
    fileSize = result.info.size
    width = result.info.width
    height = result.info.height
  } else {
    uploadBuffer = originalBuffer
    finalExt = originalExt
    fileSize = uploadedFile.size
  }

  // Build storage path with hierarchy when scope IDs are provided
  const brand = payload.brand_id ? await Brand.findOrFail(payload.brand_id) : null
  const campaign = payload.campaign_id ? await Campaign.findOrFail(payload.campaign_id) : null

  const pathSegments: string[] = []
  if (brand) {
    pathSegments.push(brand.workspaceId, brand.id)
    if (campaign) pathSegments.push(campaign.id)
    if (payload.creative_group_id) pathSegments.push(payload.creative_group_id)
    if (payload.creative_id) pathSegments.push(payload.creative_id)
  } else {
    pathSegments.push('general')
  }
  pathSegments.push(`${cuid()}.${finalExt}`)
  const storagePath = pathSegments.join('/')

  // Upload to BunnyCDN
  await BunnyStorage.upload(storagePath, uploadBuffer)
  const cdnUrl = BunnyStorage.getPublicUrl(storagePath)

  // Create Asset DB row when brand context is available
  if (brand) {
    try {
      await Asset.create({
        name: uploadedFile.clientName.slice(0, 255),
        type: fileType,
        brandId: brand.id,
        campaignId: campaign?.id ?? null,
        creativeGroupId: payload.creative_group_id ?? null,
        creativeId: payload.creative_id ?? null,
        workspaceId: brand.workspaceId,
        storagePath,
        cdnUrl,
        thumbnailUrl: fileType === 'image' ? cdnUrl : null,
        fileSize,
        mimeType: shouldConvertToWebp ? 'image/webp' : (uploadedFile.headers['content-type'] || null),
        metadata: width && height ? { width, height } : null,
        uploadedBy: auth.user?.id ?? null,
      })
    } catch (err) {
      // Clean up orphaned Bunny file
      try { await BunnyStorage.delete(storagePath) } catch {}
      throw err
    }
  }

  return response.created({
    data: {
      url: cdnUrl,
      clientName: uploadedFile.clientName,
      filename: storagePath,
      type: fileType,
      size: fileSize,
      extname: finalExt,
      ...(width && height ? { width, height } : {}),
    },
  })
}

[ ] Step 3: Clean up unused imports

Remove Drive from imports if no other method uses it. Keep Env, CloudFlare, Bunny (video), CreativeGroupCreative if other methods in the controller still reference them.

Task 5: Backend tests for CreativeAssetsController

Files:

Create: tests/functional/creative_assets/upload.spec.ts
[ ] Step 1: Write test file

typescript

import { test } from '@japa/runner'
import sinon from 'sinon'
import Asset from 'App/Models/Asset'
import { file } from '@ioc:Adonis/Core/Helpers'
import Database from '@ioc:Adonis/Lucid/Database'
import BunnyStorage from 'App/Services/BunnyStorage'
import { BrandFactory, CampaignFactory, WorkspaceFactory } from 'Database/factories'
import { accessControlTestLoginUserData } from 'TestHelpers/index'
import { RolesEnum } from 'Database/stubs/roles'

test.group('Creative Assets | upload', (group) => {
  let uploadStub: sinon.SinonStub
  let deleteStub: sinon.SinonStub

  group.each.setup(async () => {
    await Database.beginGlobalTransaction()
    uploadStub = sinon.stub(BunnyStorage, 'upload').resolves()
    deleteStub = sinon.stub(BunnyStorage, 'delete').resolves()

    return () => {
      uploadStub.restore()
      deleteStub.restore()
      return Database.rollbackGlobalTransaction()
    }
  })

  test('uploads image and converts to webp', async ({ client, assert }) => {
    const { loginUser } = await accessControlTestLoginUserData(RolesEnum.GlobalAdmin, false)

    const fakePng = await file.generatePng('10kb')
    const pngBuffer = Buffer.isBuffer(fakePng.contents)
      ? fakePng.contents
      : require('fs').readFileSync(fakePng.contents as any)

    const response = await client
      .post('/v1/assets')
      .loginAs(loginUser)
      .file('file', pngBuffer, { filename: 'photo.png' })

    response.assertStatus(201)

    const body = response.body()
    assert.equal(body.data.extname, 'webp')
    assert.equal(body.data.type, 'image')
    assert.isTrue(body.data.url.endsWith('.webp'))
  })

  test('passes through gif without webp conversion', async ({ client, assert }) => {
    const { loginUser } = await accessControlTestLoginUserData(RolesEnum.GlobalAdmin, false)

    // Create a minimal valid GIF buffer (GIF89a header)
    const gifBuffer = Buffer.from(
      'R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7',
      'base64'
    )

    const response = await client
      .post('/v1/assets')
      .loginAs(loginUser)
      .file('file', gifBuffer, { filename: 'animation.gif' })

    response.assertStatus(201)

    const body = response.body()
    assert.equal(body.data.extname, 'gif')
  })

  test('creates Asset DB row when brand_id is provided', async ({ client, assert }) => {
    const { loginUser } = await accessControlTestLoginUserData(RolesEnum.GlobalAdmin, false)
    const workspace = await WorkspaceFactory.create()
    const brand = await BrandFactory.merge({ workspaceId: workspace.id }).create()

    const fakePng = await file.generatePng('10kb')
    const pngBuffer = Buffer.isBuffer(fakePng.contents)
      ? fakePng.contents
      : require('fs').readFileSync(fakePng.contents as any)

    const response = await client
      .post('/v1/assets')
      .loginAs(loginUser)
      .field('brand_id', brand.id)
      .file('file', pngBuffer, { filename: 'logo.png' })

    response.assertStatus(201)

    const count = await Asset.query().where('brandId', brand.id).count('* as total')
    assert.equal(Number((count[0].$extras as any).total), 1)
  })

  test('does not create Asset DB row without brand_id', async ({ client, assert }) => {
    const { loginUser } = await accessControlTestLoginUserData(RolesEnum.GlobalAdmin, false)

    const fakePng = await file.generatePng('10kb')
    const pngBuffer = Buffer.isBuffer(fakePng.contents)
      ? fakePng.contents
      : require('fs').readFileSync(fakePng.contents as any)

    const beforeCount = await Asset.query().count('* as total')
    const before = Number((beforeCount[0].$extras as any).total)

    await client
      .post('/v1/assets')
      .loginAs(loginUser)
      .file('file', pngBuffer, { filename: 'quick.png' })

    const afterCount = await Asset.query().count('* as total')
    const after = Number((afterCount[0].$extras as any).total)

    assert.equal(before, after)
  })

  test('builds hierarchical storage path with full scope', async ({ client, assert }) => {
    const { loginUser } = await accessControlTestLoginUserData(RolesEnum.GlobalAdmin, false)
    const workspace = await WorkspaceFactory.create()
    const brand = await BrandFactory.merge({ workspaceId: workspace.id }).create()
    const campaign = await CampaignFactory.merge({ brandId: brand.id }).create()

    const fakePng = await file.generatePng('10kb')
    const pngBuffer = Buffer.isBuffer(fakePng.contents)
      ? fakePng.contents
      : require('fs').readFileSync(fakePng.contents as any)

    const response = await client
      .post('/v1/assets')
      .loginAs(loginUser)
      .field('brand_id', brand.id)
      .field('campaign_id', campaign.id)
      .field('creative_group_id', '999')
      .field('creative_id', '1234')
      .file('file', pngBuffer, { filename: 'bg.png' })

    response.assertStatus(201)

    // Verify BunnyStorage.upload was called with hierarchical path
    const callArgs = uploadStub.firstCall.args
    const storagePath = callArgs[0] as string
    assert.isTrue(storagePath.startsWith(`${workspace.id}/${brand.id}/${campaign.id}/999/1234/`))
    assert.isTrue(storagePath.endsWith('.webp'))
  })

  test('uses general/ prefix when no brand_id provided', async ({ client, assert }) => {
    const { loginUser } = await accessControlTestLoginUserData(RolesEnum.GlobalAdmin, false)

    const fakePng = await file.generatePng('10kb')
    const pngBuffer = Buffer.isBuffer(fakePng.contents)
      ? fakePng.contents
      : require('fs').readFileSync(fakePng.contents as any)

    await client
      .post('/v1/assets')
      .loginAs(loginUser)
      .file('file', pngBuffer, { filename: 'quick.png' })

    const callArgs = uploadStub.firstCall.args
    const storagePath = callArgs[0] as string
    assert.isTrue(storagePath.startsWith('general/'))
  })
})

[ ] Step 2: Run tests

bash

node ace test --files="tests/functional/creative_assets/upload.spec.ts"

[ ] Step 3: Also run existing asset library tests to verify nothing broke

bash

node ace test --files="tests/functional/asset_library/*"

Task 6: Frontend -- FileUploader sends scope IDs

Files:

Modify: src/components/common/FileUploader.vue
Modify: src/services/chatbotService.js
[ ] Step 1: Add scopeIds prop to FileUploader

In the props section of FileUploader.vue, add:

typescript

scopeIds: {
  type: Object,
  default: null,
},

[ ] Step 2: Send scope IDs in uploadFile()

In the uploadFile() method, after the resize fields, add:

typescript

if (this.scopeIds) {
  if (this.scopeIds.brandId) formData.append('brand_id', this.scopeIds.brandId)
  if (this.scopeIds.campaignId) formData.append('campaign_id', this.scopeIds.campaignId)
  if (this.scopeIds.creativeGroupId) formData.append('creative_group_id', this.scopeIds.creativeGroupId)
  if (this.scopeIds.creativeId) formData.append('creative_id', this.scopeIds.creativeId)
}

[ ] Step 3: Pass scopeIds from builder config components

The components that use FileUploader in the builder all have access to the Vuex store. The scope IDs come from the creative's route params.

In each builder component that uses <FileUploader>, add :scope-ids="assetScopeIds" and provide the computed:

typescript

computed: {
  assetScopeIds() {
    const props = this.$store.state.builder.creativeData?.creativeSettings?.creativeProperties
    if (!props) return null
    return {
      brandId: props.brandId,
      campaignId: props.campaignId,
      creativeGroupId: props.creativeGroupId,
      creativeId: this.$store.state.builder.creativeData?.id,
    }
  },
}

Files to update (search for <FileUploader in builder context):

src/pages/Chatbots/components/BuilderVisuals/Configuration/components/FontSettings/TypefaceSection.vue
src/pages/Chatbots/components/BuilderVisuals/Configuration/configs/ArConfiguration.vue
src/pages/Chatbots/components/BuilderVisuals/Configuration/components/VideoSection.vue
src/pages/Chatbots/components/CavaiFlow/operators/ChangeVideoOp.vue
src/pages/Chatbots/components/CavaiFlow/operators/LibraryScriptOp.vue

Also check for direct chatbotService.uploadAsset() calls outside FileUploader (e.g., in flow operators that upload images directly).

Task 7: Update AssetResource serializer

Files:

Modify: app/Resources/Asset.ts (if it exists)
[ ] Step 1: Include new columns in serialized output

Check if AssetResource explicitly lists fields. If so, add creative_group_id and creative_id to the serialization. If it uses asset.serialize() directly, the new columns will be included automatically via the model.

Task 8: Verify and commit

[ ] Step 1: Run full backend test suite

bash

npm test

[ ] Step 2: Start frontend dev server and test

Upload an image in the builder. Verify:

Image appears in asset library
URL points to BunnyCDN (not delivery server)
Storage path in Bunny has full hierarchy
WebP conversion works (upload a jpg, check the URL ends in .webp)
Fonts upload correctly (no webp conversion)
[ ] Step 3: Test backwards compatibility

If any code still calls POST /assets without brand_id, it should still work (uploads to general/ on Bunny, no DB row).

[ ] Step 4: Commit backend and frontend separately

bash

# Backend
git add -A
git commit -m "Migrate builder uploads to BunnyCDN with WebP conversion and Asset DB rows"

# Frontend
git add -A
git commit -m "Send scope IDs with asset uploads for asset library tracking"

Asset Storage Unification ​

Context ​

Out of scope ​

Architecture ​

Task 1: DB Migration -- add creative scope columns ​

Task 2: Update StoreAssetValidator ​

Task 3: Update AssetLibraryController.store() for new columns ​

Task 4: Migrate CreativeAssetsController to BunnyCDN ​

Task 5: Backend tests for CreativeAssetsController ​

Task 6: Frontend -- FileUploader sends scope IDs ​

Task 7: Update AssetResource serializer ​

Task 8: Verify and commit ​

Asset Storage Unification

Context

Out of scope

Architecture

Task 1: DB Migration -- add creative scope columns

Task 2: Update StoreAssetValidator

Task 3: Update AssetLibraryController.store() for new columns

Task 4: Migrate CreativeAssetsController to BunnyCDN

Task 5: Backend tests for CreativeAssetsController

Task 6: Frontend -- FileUploader sends scope IDs

Task 7: Update AssetResource serializer

Task 8: Verify and commit